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Executive  Summary 


Background 

The  acquisition  of  training  and  training  systems  in  support  of  major  Defense 
programs  has  been  found  to  be  an  increasingly  important  aspect  of  the  Department  of 
Defense  (DoD)  acquisition  process.  When  considered  over  the  full  acquisition  life  cycle 
of  the  Major  Defense  Acquisition  Programs  (MDAPs),  training  and  training  systems  can 
increase  operational  effectiveness  and  provide  significant  cost  efficiencies.  This  report 
addresses  key  topics  that  were  raised  in  the  Strategic  Plan  for  the  Next  Generation  of 
Training  for  the  Departmen  t  of  Defense,  prepared  by  the  Office  of  the  Deputy  Assistant 
Secretary  of  Defense  (Personnel  and  Readiness)  and  signed  by  the  Deputy  Secretary  of 
Defense  on  23  September  2010.  This  research  effort  expands  on  the  Phase  I  research, 
which  addressed  how  systems  training  considerations  are  essential  for  reaching  future 
training  goals  in  an  effective  and  timely  manner.  Although  evidence  has  existed  for  many 
years  regarding  the  importance  of  training  in  acquisition,  we  highlight  five  well-known 
programs  to  document  the  need  for  training  planning  through  the  system  life  cycle.  These 
specific  cases  were  identified  in  the  Phase  I  MDAP  report  (see  Table  ES-1).  Here,  we 
discuss  the  value  of  considering  training  early  and  often,  as  well  as  what  may  happen 
when  such  consideration  is  absent. 


Table  ES-1.  Case  studies  analyzed. 


Weapon  System 

Rationale  for  Case  Study 

Patriot  Air  and  Missile  Defense  System 

Thirty-year  Major  Defense  Acquisition — system 
upgrades  without  sufficiently  upgrading  operator 
training. 

Future  Combat  Systems 

Major  Defense  Acquisition  Program  requiring 
training  as  a  Key  Performance  Parameter — an 
example  of  early  consideration  of  embedded 
training. 

Mine-Resistant  Ambush  Protection 

Rapid  Acquisition — operator  training  was 
provided  after  fielding. 

Husky  Mounted  Detection  System 

Rapid  Acquisition — operator  training  and  training 
devices  needed  before  fielding. 

P-8A  Poseidon 

Commercial-off-the-Shelf  Aircraft  Platform 
Acquisition — training  for  multiuse  mission. 

The  study  objective  was  to  identify  and  analyze  the  specific  benefits  of  early  and 
effective  incorporation  of  training  details  into  acquisition  programs,  particularly  those 


with  significant  human-systems  interface  requirements.  The  study  results  complement  the 
latest  DoD  acquisition  and  training  policies. 

Findings 

A  number  of  acquisition  and  readiness  issues  are  related  to  a  full  scope  of  systems 
training  across  the  lifetime  of  any  given  program.  Our  analysis  suggests  that  considering 
training  system  acquisitions  as  an  integral  part  of  every  Major  Defense  Acquisition 
Program  (MDAP)  development  should  result  in  more  effective  and  efficient  operations 
and  lower  system  life-cycle  costs.  Details  follow. 

Patriot  Air  and  Missile  Defense  System 

The  Patriot  Program,  with  more  than  20  years  of  history,  provides  a  strong  case  for 
upgrading  training  in  the  complex  programs  on  a  continuing  basis  through  the  system  life 
cycle.  Evidence  from  multiple  sources  shows  that  as  the  significant  technical  upgrades 
were  performed  over  the  years,  the  training  tools  and  learning  content  were  not  enhanced 
to  optimize  the  effectiveness  of  the  evolutionary  improvements  to  the  system.  As  a 
general  rule,  systems  upgrades  should  be  accompanied  with  corresponding  changes  to  the 
training  content  at  all  levels. 

Future  Combat  Systems 

Two  aspects  of  the  Army  FCS  program  stand  out  positively.  The  FCS  program 
included  training  as  a  Key  Perfonnance  Parameter  (KPP),  and  as  a  family  of  net-centric 
weapons  systems  it  provided  netted  infrastructure  to  host  embedded  training  across  the 
many  programs  and  aspects  of  the  system  of  systems.  The  Army  recognized  embedded 
training  as  an  integral  and  essential  capability  for  the  entire  family  of  systems.  The  FCS 
is  considered  to  be  a  good  model  of  training  development  for  future  MDAPs  for  three 
reasons:  (1)  training  was  stressed  from  the  outset — training  was  included  as  a  KPP;  (2) 
embedded  training  was  integrated  into  the  FCS  operational  hardware  and  software;  and 
(3)  material  developers  worked  with  training  developers  to  monitor  the  maturity  of 
technologies  and  incorporate  those  technologies  into  the  implementation  of  training. 

Mine-Resistant  Ambush  Protection 

The  MRAP  program  was  a  rapid  acquisition  and  therefore  not  subject  to  the 
deliberate  acquisition  planning  process  evidenced  in  our  other  case  studies.  However, 
with  the  rapid  production  and  deployment  of  the  vehicles  in  2007-2008,  the  operators 
began  to  experience  vehicle  rollovers  early  on — some  60  MRAP  mishaps  were  reported 
between  November  2007  and  June  2008,  with  over  one -half  of  those  attributed  to 
rollovers.  Lack  of  training  was  identified  in  the  narratives  for  the  most  serious  incidents 
60%  of  the  time.  The  need  to  improve  training  for  these  new  types  of  vehicles  was 


IV 


recognized  early  by  the  Joint  MRAP  program  office.  A  series  of  driver 
trainers/simulators  was  developed  as  expeditiously  as  possible,  but  the  lesson  learned 
once  again  is  to  develop  training  in  concert  with  systems  development — even  in  rapid 
acquisitions. 

Husky  Mounted  Detection  System 

The  HMDS  is  another  rapid-development  program  with  some  of  the  same  training 
lessons  learned  as  those  found  in  the  MRAP  program.  It  took  2  years  for  robust  HMDS 
training  to  be  developed,  with  training  development  beginning  a  full  year  after 
deployment,  and  it  took  3  years  for  the  HMDS  training  to  reach  full  effectiveness. 

P-8A  Poseidon 

The  Navy  P-8A  Poseidon  is  a  multi-mission  aircraft  replacing  the  single-mission 
design  of  the  P-3C.  The  Poseidon  is  a  modified  Boeing  737,  which  has  had  a  mature  life 
cycle  in  commercial  use  dating  back  to  the  1960s.  The  P-8A  training  simulators  are  not 
networked  to  other  platforms  to  facilitate  training  for  battle  group,  fleet,  or  joint  task 
force  operations.  The  change  in  maintenance  concept  by  the  manufacturer  resulted  in  an 
increased  load  on  Navy  training,  with  analysis  underway  to  define  the  best  courses  of 
action  for  maintenance  training. 

Recommendations 

The  case  studies  in  this  report  document  the  benefits  of  including  training  in 
acquisition  programs  and  reinforce  the  value  of  determining  comprehensive  training 
needs  over  a  program  life  cycle.  Training  simulators  and  learning  content  for  the  system 
operators  and  maintenance  personnel  should  be  developed  before  initial  introduction  of 
the  hardware  system.  Detailed  recommendations  from  this  research  are  listed  below: 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

-  Revise  Defense  and  service  acquisition  policies  to  require  earlier 
assessments  of  training. 

•  Training  assessments  should  be  early  and  continual. 

•  System  upgrades  require  training  upgrades;  improvements  in  systems  hardware 
and  software  must  be  accompanied  by  corresponding  training  enhancements  at  all 
organizational  levels. 

•  All  MDAPs  (including  rapid  acquisitions)  should  address  training  as  an  integral 
part  of  the  acquisition  program. 

•  Impacts  to  training  concepts  and  systems  and  devices  should  be  part  of  the 
weapon  systems  development  decisions. 
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•  Integrate  embedded  training  with  MDAP  weapons  systems  as  feasible.  Net¬ 
centric  systems  of  the  future  provide  special  opportunities  to  build  in  embedded 
training. 

•  Monitor  technical  maturity  to  incorporate  parallel  critical  training  technologies. 

•  Provide  training  with  the  fielded  system  as  a  package  to  realize  the  full  capability 
of  new  systems. 

-  Training  plans  should  be  developed  well  before  successive  program 
milestones  leading  to  the  initial  fielding  of  the  hardware  system,  and 
similarly  updated  and  packaged  with  every  fielded  system  modification. 

•  Require  MDAPs  to  provide  network  connectivity  and  logical  interoperability  for 
the  suite  of  simulators  and  training  tools  so  training  scenarios  are  fully  supported 
and  integrated  with  the  live,  virtual,  and  constructive  training  environments. 

•  Synchronize  scheduling  of  simulators,  training  tools,  and  packages  to  be  available 
to  train  for  first  deployment  to  ensure  full  system  capabilities  are  available  for 
operations. 
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1.  Introduction  and  Background 


A.  Background 

The  acquisition  of  training  and  training  systems  in  support  of  major  programs  has 
been  found  to  be  an  increasingly  important  aspect  of  the  Department  of  Defense  (DoD) 
acquisition  process.  Over  the  full  acquisition  life  cycle  of  major  programs,  the  training 
systems  considerations  can  increase  operational  effectiveness  and  provide  significant  cost 
efficiencies.  The  premise  of  this  study  is  that  a  more  thorough  understanding  of  systems 
training  considerations  early  in  the  acquisition  process  will  produce  significant 
operational  effectiveness  benefits  over  the  system  life  cycle.  An  understanding  of  the  full 
spectrum  of  training  and  training  support  may  influence  major  program  decisions. 
Determining  the  benefits  and  costs  of  modern  simulators  and  simulations  requires 
considering  the  full  range  of  options  for  individual,  classroom,  and  unit  training — 
beginning  in  the  concept-development  stage  and  extending  through  the  life  of  a  major 
system. 

This  research  addresses  key  topics  that  were  raised  in  the  Strategic  Plan  for  the  Next 
Generation  of  Training  for  the  Department  of  Defense,  prepared  by  the  Office  of  the 
Deputy  Assistant  Secretary  of  Defense  (Personnel  and  Readiness)  and  signed  by  the 
Deputy  Secretary  of  Defense  on  23  September  2010.  This  next-generation  training 
strategy  provides  guidance  and  outlines  issues  for  the  future  of  Defense  training.  The 
research  effort  outlined  in  this  document  expands  on  the  Phase  I  research  to  address  how 
systems  training  considerations  are  essential  for  reaching  future  training  goals  in  an 
effective  and  timely  manner.  This  builds  on  Analysis  of  Systems  Training  Impact  for 
Major  Defense  Acquisition  Programs  (MDAPs),  Phase  I  (completed  in  July  2011)  was  to 
collect  and  process  several  decades  of  major  systems  data  to  illuminate  the  role  of 
training  in  optimizing  total  systems  performance  for  acquisition  programs  (Wisher,  et  al. 
2011). 

The  Phase  I  report  reviewed  more  than  4,000  technical  documents  and  500 
Government  Accountability  Office  (GAO)  reports  for  inclusion  in  a  database  for  future 
study.  This  research  examined  an  extensive  list  of  reports  and  studies  relating  to  systems 
acquisition  for  MDAPs  and  made  the  infonnation  available  for  future  reference  by 
creating  an  automated  database  of  common  variables.  Two  central  research  questions 
were  addressed: 
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•  Do  the  warfighters  (operators,  maintainers,  and  leaders)  and  the  acquisition 
community  benefit  from  early  consideration  of  systems  training  in  major 
acquisitions? 

•  Does  early  system  training  contribute  to  initial  readiness  and  full  use  of  a 
system’s  capability  upon  initial  delivery? 

The  DoD  spends  billions  of  dollars  each  year  developing  and  procuring  major 
weapon  systems,  and  these  expenditures  have  produced  the  most  technologically 
advanced  weapon  systems  in  the  world.  But  the  process  through  which  systems  are 
determined  and  acquired  has  often  proven  to  be  costly  and  inefficient,  as  reports  by  the 
DoD  Inspector  General  and  the  GAO  (documented  in  Phase  I)  have  repeatedly  recounted. 

The  Phase  I  study  findings  documented  results  of  a  gap  analysis  that  searched  for 
differences  between  an  “existing  status”  and  a  “potential  status”  for  what  might  be  the 
desired  status.  Four  types  of  gaps  were  identified: 

•  Knowledge  gap — a  best  training  practice  has  not  been  fully  validated  or  proven 
for  application  for  a  given  training  system. 

•  Awareness  gap — a  best  training  practice  is  proven,  established,  and  relevant,  but 
has  not  been  applied  for  the  training  system  of  interest. 

•  Implementation  gap — a  valid  best  training  practice  has  been  identified  and 
attempted,  but  did  not  work  properly  in  the  case  of  a  given  training  system. 

•  Commitment  gap — a  valid  best  training  practice  is  recognized  but  not  applied  due 
to  policy,  cost,  schedule  or  other  factors. 

The  Phase  I  analysis  reported  26  instances  of  gaps,  validating  the  need  for  a  more  critical 
treatment  of  training  in  major  acquisitions.  The  gap  and  trend  analysis  of  these  data 
provide  a  starting  point,  namely  that  much  was  known  about  training  but  not  applied  in 
the  acquisition  of  a  particular  system  and  this  was  a  persistent  and  continuous  hindrance 
to  overall  systems  performance.  The  second  phase  of  the  study  examines  a  series  of 
selected  systems  more  closely  for  detailed  training  assessment  through  a  case-study 
methodology. 

The  present  (Phase  II)  research  builds  on  the  Phase  I  report  findings,  database,  and 
methodology  and  on  the  Institute  for  Defense  Analyses  (IDA)  staff  experience  with  DoD- 
wide  training  and  large  acquisition  program  issues.  In  this  study,  five  specific  cases 
identified  in  Phase  I  were  selected  for  detailed  analysis  (see  Table  1-1). 
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Table  1-1. 

Case  studies  analyzed. 

Weapon  System 

Rationale  for  Case  Study 

Patriot  Air  and  Missile  Defense  System 

Thirty-year  Major  Defense  Acquisition — system 
upgrades  without  sufficiently  upgrading  operator 
training. 

Future  Combat  Systems 

MDAP  requiring  training  as  a  Key  Performance 
Parameter — an  example  of  early  consideration  of 
embedded  training. 

Mine-Resistant  Ambush  Protection 

Rapid  Acquisition — operator  training  was 
provided  after  fielding. 

Husky  Mounted  Detection  System 

Rapid  Acquisition — operator  training  and  training 
devices  needed  before  fielding. 

P-8A  Poseidon 

Commercial-off-the-Shelf  Aircraft  Platform 
Acquisition — training  for  multiuse  mission. 

B.  Objectives 

The  study  objective  is  to  identify  and  analyze  the  specific  benefits  of  early  and 
effective  incorporation  of  training  details  into  acquisition  programs,  particularly  those 
with  significant  human-systems  interface  requirements.  The  intent  was  to  provide 
evidence  from  acquisition  programs  to  substantiate  our  study  premise  regarding  more 
detailed  training  assessments  and  planning  earlier  in  the  acquisition  process.  The  research 
provides  an  empirical  base  for  the  DoD  training  and  acquisition  communities  to 
understand  and  justify  the  need  to  plan  for  efficient  and  effective  operator,  maintainer, 
and  leadership  training  early  and  continually  in  the  acquisition  process  to  improve  the 
resulting  capabilities  and  utility  of  future  complex  weapon  systems.  The  study  results 
complement  the  latest  DoD  acquisition  and  training  policies. 

C.  Organization  of  Report 

Chapters  2-6  correspond  to  the  five  case  studies.  Each  case  study  was  written  by  an 
author  (or  authors)  with  relevant  expertise.  Chapter  7  discusses  the  general  implications 
to  be  drawn  from  the  case  studies  that  apply  to  MDAP  acquisition  processes  in  general. 
Appendix  A  provides  brief  resumes  of  the  case  study  authors.  Appendix  B  gives  a 
definition  of  nine  Technology  Readiness  Levels  as  they  apply  to  the  Future  Combat 
Systems  (FCS).  Appendix  C  lists  the  figures  and  tables  found  in  the  report,  Appendix  D 
lists  the  references,  and  Appendix  E  lists  the  acronyms. 
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2.  Patriot  Air  and  Missile  Defense  System 

— Robert  A.  Wisher 


A.  Patriot  Background 

The  Patriot  system  began  because  of  the  need  to  replace  an  aging  and  limited  air 
defense  system  in  the  1970s,  the  Nike-Hercules,  and  augment  another,  the  Hawk,  with 
one  that  can  defend  against  higher  altitude  threats  and  do  so  at  ever-increasing  ranges. 
The  Patriot  case  study  focuses  on  gaps  between  the  upgrades  to  the  technical  capabilities 
of  a  system  and  the  upgrades  to  the  training  necessary  for  effective  operation.  It 
highlights  the  consequences  of  pursuing  substantial  increases  to  a  system’s  capabilities 
while  delivering  only  marginal  increases  to  an  operator’s  capabilities  through  training, 
relying  instead  on  automated  procedures  to  offset  the  shortcoming. 

The  Air  Land  Battle  doctrine,  published  in  1982  as  Operations  (FM  100-5), 
represented  the  U.S.  Army  warfighting  formula  (US  Army  1982).  It  called  for  an 
extended  battlefield  and  a  close  interaction  between  air  and  ground  capabilities.  Air 
Defense  became  one  of  five  critical  tasks  of  the  central  battle,  and  with  it  came  the  need 
to  counter  emerging  threats  and  changing  operating  environments,  from  defeating  air- 
breathing  threats  to  defeating  tactical  ballistic  missiles.  This  shift  in  warfighting 
formulation  required  upgrades  to  the  technical  capabilities  of  many  weapon  systems, 
including  the  Patriot  Air  and  Missile  Defense  System,  and  should  have  ben 
complemented  by  upgraded  training  technologies  to  match. 

The  Patriot  is  a  surface-to-air  missile  system  having  the  primary  mission  to  function 
as  the  Anny’s  anti-ballistic-missile  system.  This  case  study  traces  how  the  program’s 
well-intended  but  single-minded  objective  to  extend  system  capabilities  led  to  later 
problems  with  overall  optimal  system  performance.  Principal  factors  for  this  were 
human-performance  and  operator-training  issues  that  accumulated  but  were  not  resolved 
along  the  way.  The  Patriot  system  evolved  with  technical  enhancements  over  a  period  of 
more  than  two  decades,  but  operators  lacked  the  rigorous  training  to  deal  with  this 
different  role — a  role  as  supervisor  of  automated  functions  and  services  subordinate  to 
the  operator.  In  parallel  with  the  technical  enhancements,  the  manner  in  which  operators 
controlled  the  system  also  evolved,  migrating  from  a  manual-engagement  mode  to  one  in 
which  operators  became  supervisors  of  a  set  of  automated-control  systems.  The 
operators,  who  remain  the  ultimate  decision-makers,  needed  to  deal  with  new  technical 
features  and  information  sources  manifested  through  higher  levels  of  automation.  As  this 
case  study  illustrates,  the  lack  of  proper  job  task  analysis,  the  lack  of  job  task  re-analysis 
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during  or  after  significant  upgrades,  and  reliance  on  training  devices  that  focused  on 
narrow  training  scenarios  and  rote  training  methods  had  serious  aftereffects.  Policies  on 
personnel  assignment  that  impeded  the  development  of  needed  expertise  were  another 
factor.  A  high  rate  of  fratricides  during  Operation  Iraqi  Freedom  led  to  boards  of  inquiry 
and  demands  for  improvements  to  operator  training.  This  chapter  also  considers  that  the 
consequences  of  improper  job  task  analysis  and  limited  training  may  be  repeated  with  an 
even  more  capable  system,  the  THAAD  (Terminal  High  Altitude  Air  Defense,  formerly 
Theater  High  Altitude  Air  Defense).  This  case  study  also  offers  a  general 
recommendation  on  policies  to  upgrade  training  when  technical  capabilities  are  being 
upgraded. 

This  study  first  introduces  the  Patriot  system  and  the  defined  role  of  operators.  The 
study  then  covers  the  organizational  structure  supporting  its  operation,  a  depiction  of 
upgrades  to  the  system,  training  practices,  early-warning  signs  regarding  inadequate 
training,  field  performance,  and  calls  to  upgrade  the  training  years  later. 

B.  Patriot  System  Overview 

The  Patriot  (Figure  2-1)  is  a  mobile  air  and  missile  defense  system  that  counters 
missile  and  aircraft  threats.  It  offers  commanders  the  capability  to  defend  deployed  forces 
and  critical  assets  from  missile  and  aircraft  attack  and  to  defeat  enemy  surveillance 
assets,  including  unmanned  aerial  vehicles,  in  a  variety  of  weather  conditions  and 
countermeasure  environments.  This  was  not  the  mission  originally  envisioned.  In  1964, 
the  Secretary  of  Defense  directed  that  the  Army  Air  Defense  System  for  the  1970s 
assume  the  name  Surface-to-Air  Missile,  Development  (SAM-D).  With  the  successful 
engagement  of  a  drone  in  1975,  SAM-D  was  renamed  the  Patriot  Air  Defense  Missile 
System.  Patriot  is  actually  an  acronym  for  Phased  Array  Tracked  Radar  Intercept  of 
Target.  Full-scale  development  of  the  system  began  in  1976,  with  deployment  in  1984. 
The  Patriot  is  employed  in  the  field  through  a  battalion  echelon  organizational  structure, 
which  includes  a  headquarters  battery;  a  maintenance  company;  and  four  to  six  line 
batteries,  each  firing  battery  consisting  of  three  or  four  platoons.  The  basic  building  block 
of  a  Patriot  unit,  the  firing  battery  includes  eight  missile  launchers,  radar  and 
communications  equipment,  and  the  engagement  control  station  (ECS).  From  an 
operator’s  training  perspective,  the  ECS  is  one  of  the  most  critical  components,  and  the 
only  manned  station  in  a  Patriot  fire  unit.  It  is  tasked  with  monitoring  readiness,  threat¬ 
ordering,  and  giving  priorities  to  radar,  among  other  responsibilities. 

The  ECS  includes  the  weapons-control  computer;  the  data-link  terminal;  and  work 
stations  for  the  three  operators,  a  Tactical  Control  Officer  and  a  Tactical  Control 
Assistant,  both  trained  on  Patriot  tactics,  who  together  control  the  air  battle  for  their 
battery,  along  with  one  communications  operator.  The  system  is  operated  through  a  user 
interface  (screen,  keyboard,  isometric  stick,  and  switch  indicators)  in  concert  with  the 
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Patriot  user  interface  software.  The  Tactical  Control  Officer  is  responsible  for  all 
identification  judgments  and  for  launching  missiles  at  the  tracks  that  are  to  be  engaged  by 
Patriot.  The  Tactical  Control  Assistant  aids  the  Tactical  Control  Officer  by  supplying 
information  about  incoming  tracks.  The  ECS  communicates  to  the  launching  stations, 
other  Patriot  batteries,  and  the  higher  command  headquarters. 


Source:  Raytheon  photo  gallery: 

http://www.ravtheon.com/businesses/rids/businesses/patriot/patriot  amdp/index.html. 

Figure  2-1.  Patriot  firing. 

The  ECS  is  capable  of  operating  in  either  an  autonomous  mode,  that  is,  as  a  stand¬ 
alone  facility  using  its  own  radar  and  making  its  own  firing  decisions,  or  in  centralized 
mode,  that  is,  in  combination  with  the  ECSs  of  up  to  five  other  batteries  to  fonn  a 
battalion  under  the  command  of  the  Information  and  Coordination  Central  (ICC).  The 
ICC  is  similar  to  the  ECS  in  general  appearance  and  features  but  monitors  a  wider  sector 
of  operations  and  directs  the  activity  of  subordinate  ECSs  via  voice  and  digital  data  links 
when  the  battalion  is  operating  in  centralized  mode.  The  other  components  of  a  fire  unit 
are  the  radar  set,  the  antenna  mast  group,  the  power  plant,  and  the  launchers.  Additional 
personnel  are  employed  to  handle  repair,  refuel,  and  reload  tasks. 

1.  System  Upgrades 

As  with  other  weapon  systems,  the  Patriot  needed  additional  capabilities  to  keep 
pace  with  changing  threat  environments  and,  for  example,  the  demands  of  joint¬ 
warfighting  doctrine,  such  as  precision  engagement.  The  program  sponsors  of  the  Patriot 
also  desired  to  take  advantage  of  technological  innovation  and  information  superiority  to 
enhance  overall  performance.  In  particular,  the  need  to  defend  against  tactical  ballistic 
missiles  strained  the  initial  capabilities,  requiring  a  series  of  Patriot  Advanced  Capability 
packages,  implemented  through  a  series  of  stand-alone  fielding  configurations.  Each 
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configuration  consists  of  a  grouping  of  materiel-change  packages  and  a  software  upgrade 
called  a  post-deployment  build. 

The  Patriot  system  had  significant  upgrades  over  the  past  30  years.  Specific  details 
of  operational  tests  and  limited  user  tests  are  omitted  unless  there  was  an  immediate 
impact  on  training,  which  will  be  covered  in  a  later  section  on  MANPRINT  (Manpower 
and  Personnel  Integration)  assessments.  MANPRINT  is  a  management  and  technical 
program  concerned  with  the  integration  of  human  considerations  into  systems 
acquisition. 

Full-scale  development  of  the  Patriot  began  in  1976  with  the  MIM-104  Patriot.  The 
baseline  system  combined  several  new  technologies  at  the  time,  notably  the  phased-array 
radar  and  the  track-via-missile  guidance  system.  During  the  1980s,  the  Patriot  was 
upgraded  in  minor  ways,  primarily  software  upgrades.  Its  modular  nature  has  made 
upgrades  to  its  hardware  and  software  a  relatively  straightforward  engineering  effort,  one 
that  was  almost  continuous  in  nature  over  the  years.  Nearly  every  major  subcomponent 
has  been  upgraded,  with  the  most  common  upgrades  to  the  software  and  the  missile  itself. 
Figure  2-2  is  the  prime  contractor’s  (Raytheon)  chart  of  these  advances  to  the  Patriot’s 
capability  over  a  10-year  period,  followed  by  a  brief  description. 
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Figure  2-2.  Upgrades  to  the  Patriot,  1990  to  2000. 


There  have  been  five  major  variants  of  the  Patriot  system  from  its  inception  to  the 
present  time: 
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•  Variant  1 — The  MIM-104A,  considered  the  standard  missile,  was  optimized  for 
engagements  against  aircraft,  with  limited  capability  against  ballistic  missiles, 
with  an  operational  range  of  70  km. 

•  Variant  2 — Patriot  Advanced  Capability,  PAC-1  was  the  1988  upgrade.  The 
MIM-104B  was  a  slight  variation,  with  the  added  capability  to  seek  and  destroy 
electronic  countenneasure  emitters  and  with  an  operational  range  of  70  km. 

•  Variant  3 — Patriot  Advanced  Capability,  PAC-2.  This  was  the  Patriot’s  first 
major  missile  upgrade,  with  the  MIM-104C  missile.  The  upgrade  sought  a 
capability  to  reliably  destroy  inbound  ballistic  missiles.  PAC-2  was  deployed  to 
units  in  1990.  The  operational  range  was  extended  to  160  km. 

•  Variant  4 — PAC-2/Guidance  Enhanced  Missile.  Here,  the  missiles  were  enhanced 
significantly  with  respect  to  software  and  warhead  detonation.  Four  versions  of 
the  Guidance  Enhanced  Missile  were  introduced  in  the  1990s  and  early  2000s. 

•  Variant  5 — Patriot  Advanced  Capability,  PAC-3.  This  is  considered  a  major 
system  improvement  over  the  PAC-2,  with  a  new  hit-to-kill  interceptor  missile, 
along  with  improved  communications,  radar,  and  ground  support  systems.  Full- 
scale  production  began  in  late  2002. 

2,  Recent  Patriot  Activity 

The  Army  conducted  one  major  developmental  Patriot  flight-test  mission  and  a 
Post-Deployment  Build-7  Developmental  Test  and  Evaluation  in  FY11.  The  Army 
conducted  three  major  developmental  Patriot  flight-test  missions  in  early  FY12.  The 
Medium  Extended  Air  Defense  System  (MEADS)  was  intended  to  be  a  more  deployable, 
mobile,  and  capable  air-  and  missile-defense  system  than  Patriot.  The  DoD  has  decided 
not  to  field  MEADS,  although  it  will  continue  program  development  through  the 
developmental  phase  of  the  program. 

C.  Patriot  Training  Systems  Development 

Job  training  for  a  Patriot  fire-control  enhanced  operator,  who  can  serve  in  the  ECS 
as  the  Tactical  Control  Assistant,  requires  10  weeks  of  Basic  Combat  Training  and  20 
weeks  of  Advanced  Individual  Training  with  on-the-job  instruction.  Part  of  this  time  is 
spent  in  the  classroom  and  in  the  field  under  simulated  combat  conditions,  concentrating 
on  ready-for-action  drills  and  how  to  conduct  fundamental  commands  given  by  their 
tactical  control  officer.  Training  continues  after  assignment  to  an  operational  unit,  with 
training  aids  and  devices  such  as  the  troop  proficiency  trainer  (TPT),  an  early  example  of 
embedded  training.  Also  in  the  ECS  is  an  Air  Defense  Artillery  Officer  who  serves  as  the 
Tactical  Control  Operator.  The  officer  must  have  completed  the  Air  Defense  Artillery 
Officer  Basic  Course  and  have  knowledge  of  the  Patriot  tactics,  techniques,  and 
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procedures.  In  some  cases  a  senior  noncommissioned  officer  can  also  serve  as  Tactical 
Control  Officer. 

Training  in  the  unit  is  conducted  primarily  by  the  battery  trainer,  who  checks  that  all 
training  meets  the  Anny  Training  and  Evaluation  Program  and  gunnery  standards. 
Battery  trainers  receive  instruction  from  battalion  trainers,  who  ensure  the  crews  follow 
the  same  tactics,  techniques,  and  procedures  as  directed  by  the  battalion  commander.  In 
recent  years,  a  Patriot  Master  Gunner  course  was  developed  (Villa  2006).  The 
certification  of  units  is  conducted  under  the  guidance  of  the  Air  Defense  Artillery  Patriot 
Brigade  Gunnery  Program.  The  program,  which  is  based  on  a  series  of  12  gunnery  tables, 
is  designed  to  develop  and  then  test  the  proficiency  of  the  individual,  crew,  and  battery  in 
a  sequential,  perfonnance-oriented  training  environment.  Following  successful 
completion  of  the  basic  gunnery  tables  (I  through  IV)  individuals  receive  a  basic 
certification  as  a  qualified  crew  member.  This  is  achieved  through  hands-on  training, 
individual  instruction,  and  successful  completion  of  practical  and  written  exercises.  The 
intennediate  (V  through  VII)  and  advanced  gunnery  tables  (IX  through  XII)  are  the 
responsibility  of  the  parent  brigade. 

1.  The  Operator  Task 

In  the  context  of  the  current  case  study,  one  original  task  is  highlighted,  namely  the 
Perform  Air  Battle  Engagement  Task  (Anny  Task  44-1 -9002). 1  The  standard  for  the 
original  task  is  to  deny  hostile  aircraft  successful  engagement  of  high-value  assets  while 
providing  friendly  forces  maximum  protection.  It  is  assumed  that  the  Patriot  system  is 
operational,  with  the  crew  ready  to  participate  in  an  air  battle  exercises  and  that  the 
battery  is  at  designated  Alert  State.  There  are  10  performance  steps,  such  as  The 
commander  or  leader  assures  no  friendly  casualties.  Beneath  the  performance  steps  are  a 
total  of  109  sub-steps,  as  derived  from  the  task  analysis  in  the  Systems  Approach  to 
Training  model.  The  analysis  indicated  that  overall  task  of  air  battle  engagement  requires 
the  operator  to  perform  a  great  variety  of  discrete  actions,  including  monitoring, 
detecting,  recognizing,  identifying,  assigning,  establishing,  operating,  evaluating, 
applying,  verifying,  confirming,  informing,  rotating,  allowing,  ordering,  configuring, 
implementing,  directing,  ensuring,  enforcing,  reporting,  and  updating  for  the  overall  task 
of  air  battle  engagement. 

Throughout  the  perfonnance  of  the  task,  many  checks  are  made  of  the  information 
fed  through  the  operator  workstation  while  communicating  with  higher  headquarters  and 
others  in  the  ECS.  A  target  engagement  can  be  carried  out  in  manual,  semi-automatic,  or 
automatic  mode.  When  the  decision  to  engage  has  been  made  as  ordered  by  the  Air 


The  task  has  since  been  further  refined  into  a  set  of  smaller  tasks,  such  as  Perform  Friendly  Protect  at 
the  ECS,  which  roughly  correspond  to  performance  steps  in  the  original  task. 
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Defense  Artillery  Fire  Coordination  Officer,  the  launch  station(s)  is  selected  and  data  are 
transmitted  to  the  designated  missile(s). 

The  job  task  analysis  surrounding  the  crew  activities  has  not  kept  up  with  the 
hardware  and  software  system  upgrades.  Some  performance  sub-steps  in  the  original 
task,  such  as  confirm  training  in  identification  friend  or  foe  procedures,  do  not 
necessarily  address  underlying  expectations,  such  as  does  the  operator  have  adequate 
knowledge  (the  operator’s  knowledge  may  not  be  adequate  due  to  marginal  training 
performance  or  skill  decay).  Research  on  skill  decay,  for  example,  shows  a  time  course  of 
knowledge  and  skill  decay  for  step-by-step  procedural  tasks,  such  as  the  air  battle 
engagement  task,  that  is  governed  by  the  characteristics  of  the  specific  task,  such  as 
number  of  steps  and  internal  cues  (Hagman  and  Rose  1983).  Based  on  the  task 
description  for  the  original  Patriot  air  battle  engagement,  the  performance  of  operators 
will  diminish  quickly  after  a  period  of  nonuse  as  predicted  by  the  Flagman-Rose  skill 
retention  model.  This  translates  to  a  small  percentage  of  operators  in  a  unit  being  able  to 
successfully  perform  the  task  during  a  retention  interval,  or  a  time  of  nonuse. 
Furthermore,  what  might  be  successfully  retained  are  limited  capabilities  that  are  being 
practiced  and  may  not  decay  but  which  do  not  necessarily  improve  with  practice  unless 
the  training  adds  novelty  and  challenge  to  the  scenarios. 

2,  Patriot  Training  Aids  and  Devices 

Patriot  training  takes  advantage  of  simulations,  simulators,  and  various  training  aids. 
Some  are  embedded  into  the  actual  system,  others  are  housed  in  classroom  facilities,  and 
new  versions  are  available  as  tabletop  trainers.  A  brief  description  is  provided  here. 

a.  Troop  Proficiency  Trainer 

The  TPT  is  the  embedded  training  simulator  that  was  provided  with  the  original 
deployment  of  the  Patriot  system.  A  software  program  used  to  train  ICC  and  ECS 
operators,  the  TPT  is  also  used  to  evaluate  and  maintain  proficiency  levels  of  current 
operators.  TPT  can  be  programmed  for  battalion/battery  exercise  training.  It  has  a 
scenario  authoring  capability,  but  no  automated  training  features,  such  as  feedback, 
performance  feedback,  or  performance  monitoring.  The  TPT  lacks  a  demonstration 
mode,  in  which  a  training  officer,  for  example,  could  explain  target-management 
techniques.  Similarly,  it  does  not  have  a  capability  to  freeze,  playback,  or  fast-forward 
scenarios,  further  limiting  its  instructional  potential. 

b.  Patriot  Tactical  Operations  Simulator/Trainer — 1976  to  1989 

The  Tactical  Operations  Simulator/Trainer  was  a  full-task  simulator  for  the  Patriot 
system.  It  was  designed  to  simulate  the  two-person  ECS  environment.  It  originally 
accommodated  only  a  single  person  and  was  not  designed  to  accommodate  a  proficiency 
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assessment  capability,  although  the  overall  capabilities  were  enhanced  during  its  life 
cycle.  It  was  renamed  the  Recon figurablc  Tactical  Operations  Simulator  in  1989. 

c.  Patriot  Conduct  of  Fire  Trainer  (PCOFT) — 1983  to  2006 

The  PCOFT  was  a  classroom  trainer  that  used  mock-up  workstations  and  computer¬ 
generated  scenarios  to  simulate  the  battlefield.  The  PCOFT  runs  Patriot  tactical  software 
for  console  reproductions  of  the  ECS  and  ICC. 

d.  Reconfigurable  Tactical  Operations  Simulator,  1989  to  present;  as  a 
Training  device,  2007-present 

This  training  aid  has  been  in  use  since  the  late  1970s  to  support  major  exercises  and 
experimentation  in  air  defense.  The  hardware  is  commercial  off  the  shelf,  and  the  Army 
owns  the  software.  It  is  an  example  of  a  medium-  to  high-fidelity  trainer,  which  supports 
many  but  not  all  air  battle  operations  tasks,  and  is  thus  referred  to  as  a  part-task  trainer.  It 
can  be  reconfigured  to  support  other  air  defense  systems,  such  as  THAAD  and  MEADS. 
It  can  be  used  as  an  analytical,  exercise,  and  training  tool.  The  Reconfigurable  Tactical 
Operations  Simulator  supports  the  development  of  air  defense  scenarios,  scripting  of 
timed  events,  collection  and  processing  of  air  defense  events  supporting  analysis  of 
scenario  results. 

e.  Reconfigurable  Table  Top  Trainer 

This  device  replaces  the  original  PCOFT  for  institutional  training.  Workstations  are 
in  use  at  the  institutional  training  site  at  Fort  Sill,  OK,  after  a  recent  relocation  from  Fort 
Bliss,  TX,  as  part  of  the  base  realignment  and  closure  process. 

3.  Patriot  Operator  Training  Research 

In  an  assessment  of  performance  of  enlisted  Patriot  operators  to  link  perfonnance  to 
the  outcomes  of  simulated  air  battles,  a  RAND  study  examined  how  differences  in 
personnel  quality,  as  measured  by  the  Armed  Forces  Qualification  Test,  and  training 
backgrounds,  as  indicated  by  job  history  and  training  assignments,  affect  the  execution  of 
Tactical  Control  Assistant  functions  and  the  outcomes  of  battles  (Orvis,  Childress  and 
Polich  1992).  From  late  1988  to  mid-1999,  virtually  all  the  Army’s  enlisted  Patriot 
operators  (Military  Occupational  Specialty  24T  at  the  time)  were  tested  on  a  Patriot 
Conduct  of  Fire  Trainer,  a  device  that  simulates  fully  interactive  battalion  operations. 
Four  simulated  air  battles  were  developed  for  the  assessment,  along  with  a  written  test 
measuring  knowledge  of  tactical  operating  procedures.  A  key  finding  demonstrated  that  a 
one  Mental  Category  change  in  Armed  Forces  Qualification  Test  scores  equaled  or 
surpassed  the  effect  of  a  year  of  operator  experience  or  of  frequent  training.  Soldiers 
learned  several  key  tactical  skills  shortly  after  Advanced  Individual  Training,  including 
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the  ability  to  decide  whether  and  when  to  engage  aircraft  based  on  the  nature  and  severity 
of  threat.  The  study  provides  evidence  linking  training  in  units  with  success  during  air 
battles. 

Human  factor  elements  of  the  system  were  investigated  during  Patriot’s  second 
operational  test  (Carter  and  Lockhart  1982).  Questionnaires  addressing  specific  test 
issues  were  developed,  checklists  were  completed,  and  interviews  conducted  with  the  85 
service  members  who  participated  in  the  operational  test.  One  objective  of  the  interviews 
was  to  determine  whether  problems  existed  with  the  troop  proficiency  trainer  programs. 
For  the  items  related  to  the  four  TPT  software  programs,  all  were  rated  poor  to  fair  by  the 
ICC  personnel.  A  summary  of  open-ended  comments  from  the  ICC  and  ECS  personnel 
indicated  the  problems: 

The  TPT  tapes  were  not  programmed  correctly.  Many  errors  and  problems 
were  identified.  They  included  the  signaling  of  wrong  targets  as  being 
hooked,  alert  messages  remaining  on  the  screen,  and  slow  presentation  of 
pop-up  targets... Crewmen  stated  that  the  tapes  could  not  be  utilized  for 
training  console  operators  and  as  an  evaluative  or  diagnostic  tool.  They 
also  reported  that  the  firing  doctrine  had  not  been  correctly  incorporated 
into  the  TPT  tapes.  The  service  members  were  not  provided  with 
guidelines  regarding  the  use  and  function  of  the  tapes  and  taught  how  to 
use  them  for  training.  (Carter  and  Lockhart  1982,  p.  9) 

The  military  services  were  interested  in  the  construct  of  embedded  training,  offered 
by  the  early  TPT,  which  is  training  provided  by  capabilities  built  into  or  added  onto 
operational  systems.  In  an  early  tri-service  review  of  the  components  of  embedded 
training,  Warm  et  al.  (1988)  observed  that  a  number  of  the  embedded-training 
components  studied  relied  on  pre-defined  scenarios  to  provide  practice  and  training 
opportunities.  Generally,  a  limited  number  of  pre-defined  scenarios  are  available  for 
presentation  through  the  embedded  training  component.  Interview  data  suggested  that  the 
small  number  of  scenarios  available  in  some  cases,  notably  the  Patriot  TPT,  restricted  the 
quality  and  amount  of  training  that  could  be  provided.  In  the  case  of  the  Patriot  TPT,  this 
leads  to  a  situation  where,  after  a  number  of  exposures  to  each  scenario,  trainees  “learn 
the  scenarios” — that  is,  learn  where  and  when  in  an  exercise  to  expect  particular  targets 
to  appear.  In  such  a  situation,  the  training  value  of  the  scenarios  is  quickly  lost,  since  the 
trainee  is  responding  to  a  known  situation  rather  than  responding  to  a  new  and  emerging 
situation.  One  observation  is  that  when  pre-defined  scenarios  are  to  be  used  to  implement 
embedded  training,  a  larger  number  and  variety  of  scenarios  than  is  provided  in  the 
systems  studied  may  be  necessary  to  ensure  effective  sustainment  training. 

There  is  a  small  body  of  research  on  the  perfonnance  of  Patriot  operators  executing 
the  air  battle  engagement  task.  One  study  reported  a  considerable  range  in  the  accuracy  of 
both  Tactical  Control  Officers  and  Tactical  Control  Assistants  for  tracks  with  conflicting 
information  (Adehnan,  Tolcott  and  Bresnick  1993).  Conflict  occurs  when  some 
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information  or  “cues”  indicate  that  the  aircraft  is  friendly  and  that  other  information 
indicates  that  it  is  hostile.  In  a  follow-on  academic  study  of  28  Patriot  Tactical  Control 
Officer/Tactical  Control  Assistant  teams,  scenarios  were  programmed  into  the  Patriot 
simulator,  which  varied  the  number  of  tracks  on  the  display  and  degree  of  conflicting 
information  (Adehnan,  Christian,  et  al.  1997).  The  experiment  examined  the  effect  of 
communication  training  on  team  perfonnance.  The  communication  training  did  not 
improve  team  performance;  rather,  it  was  the  number  of  hours  the  team  had  previously 
worked  together  that  influenced  team  performance.  But  this  effect  held  for  only  the  tasks 
for  which  the  Patriot  team  routinely  trains.  Important  for  the  current  case  study,  the  effect 
did  not  transfer  to  the  infrequent  and  more  cognitively  stressing  tasks  where  there  is 
conflicting  information  about  unknown  aircraft. 

Other  research  with  Patriot  operators  found  significant  order  effects  for  difficult 
identification  tasks.  An  order  effect  occurs  when  different  identification  judgments  or 
different  engagement  decisions  are  made  depending  on  the  sequence  in  which  the  same 
information  or  cues  are  presented  to  the  operator.  In  particular,  order  effects  were  found 
for  tasks  with  conflicting  information  about  an  incoming  aircraft  (Adehnan,  Bresnick,  et 
al.  1996).  This  body  of  research,  sponsored  in  part  by  the  U.S.  Army  Research  Institute, 
was  conducted  after  Operation  Desert  Storm  but  before  Operation  Iraqi  Freedom. 

D.  Patriot  Training  Systems  Issues 

Stemming  from  the  high  fratricide  rate  during  Operation  Iraqi  Freedom,  the  Patriot 
Vigilance  project,  which  was  called  at  the  request  of  the  Fort  Bliss  Commander, 
examined  perfonnance  and  training  issues.  During  Operation  Iraqi  Freedom,  18%  of 
engagements  resulted  in  fratricide.  Of  particular  interest  in  the  project  were  the  vigilance 
of  operators  and  their  situational  awareness  related  to  the  control  of  automated  command 
systems.  Personnel  from  the  Army  Research  Laboratory  were  invited  to  lend  their 
expertise  in  human  systems,  human  performance,  and  training  to  understand  the  nature  of 
human  errors  that  contributed  to  the  fratricide  rate.  As  part  of  a  critical  incident 
assessment,  the  staff  studied  findings  from  the  fratricide  boards  of  inquiry,  observed 
Patriot  training  and  operations,  and  interviewed  key  personnel  in  the  Air  Defense 
community.  A  supporting  technical  report  covered  the  human  perfonnance  issues 
surrounding  automated  systems  and  the  human  checks  and  balances  needed  to  ensure 
smooth  and  accurate  operations  (Hawley,  Mares  and  Giammanco  2005). 

In  a  review  of  the  Patriot  Vigilance  project,  and  a  reexamination  of  more  than  two 
decades  of  MANPRINT  assessments  and  analysis  for  the  Patriot,  Dr.  John  Hawley,  a 
research  psychologist  and  human  factors  expert  with  the  ARL  Human  Research  and 
Engineering  Directorate,  offered  observations  and  lessons  from  the  developmental 
experience.  In  2007,  Dr.  Hawley  issued  a  report  titled  Looking  Back  on  20  Years  of 
MANPRINT  on  Patriot:  Observations  and  Lessons  (Hawley  2007),  the  key  points  of 
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which  are  summarized  below  as  they  relate  to  the  current  MDAP  project.  The  ARL  report 
was  intended  not  as  a  critique  of  specific  decisions  but  rather  as  a  reflection  on 
institutional  learning  and  practice  improvement. 

MANPRINT  is  an  Army  program  initiated  in  the  1980s  to  apply  a  “systems 
approach”  to  weapon  system  acquisition  by  providing  a  framework  that  calls  for  early 
consideration  of  key  system  planning  activities  and  development  requirements  (Booher 
1990).  MANPRINT  was  developed  in  part  as  a  consequence  of  documented  problems 
found  in  weapon  system  acquisitions  due  to  a  failure  to  properly  integrate  manpower, 
personnel,  and  training  into  the  weapon  system.  MANPRINT  is  a  comprehensive 
management  and  technical  effort  designed  to  ensure  integration  of  all  relevant 
information  from  seven  MANPRINT  domains.  Three  domains  are  relevant  to  the  current 
case  study: 

•  Personnel — The  human  aptitudes,  skills,  and  capabilities  required  to  operate, 
maintain,  and  support  a  system  in  peacetime  and  war. 

•  Training — The  instruction  and  resources  required  to  provide  [Army]  personnel 
with  requisite  knowledge,  skills,  and  abilities  to  properly  operate,  maintain,  and 
support  [Anny]  systems. 

•  Human  Factors  Engineering — The  comprehensive  integration  of  human 
capabilities  and  limitations  into  system  definition,  design,  and  evaluation  to 
promote  effective  soldier-machine  integration  for  optimal  total  system 
performance. 

1.  Twenty  Years  of  MANPRINT  on  Patriot:  Synopsis 

Patriot  evolved  over  a  two-decade  period,  during  which  it  incorporated  new  features 
and  added  characteristics  appropriate  for  future  systems.  The  system  became  increasingly 
complex  and  demanded  substantial  knowledge  to  comprehend  the  system’s  operation.  A 
logic  model  was  developed  for  the  Operation  Iraqi  Freedom  incidents  with  the  intention 
to  understand  how  the  fratricide  incidents  that  occurred  proved  to  be  almost  inevitable. 
The  three  key  factors/blocks  in  the  logic  model,  basically  a  causal  network,  are  described 
briefly  below: 

•  Undisciplined  Automation — This  is  the  consequence  of  designers  automating 
certain  functions  and  then  users  implementing  the  now  automated  functions 
without  regard  for  consequences  on  overall  human  performance.  For  the  Patriot 
system,  there  was  insufficient  attention  paid  to  whether  operators  could 
adequately  perform  residual  functions,  what  the  impact  was  on  the  reliability  of 
decisions,  and  how  operators  should  be  trained. 

•  Automation  Bias  by  Operators — A  series  of  operational  tests  revealed  that  the 
Patriot  system’s  automated  engagement  logic  was  subject  to  misclassification 
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problems,  which  in  turn  were  not  fully  addressed  during  system  software 
upgrades.  A  recognition  of  this  shortcoming  was  not  included  in  operator  training, 
and  operators  assumed  a  “fascination  with  and  blind  faith  in  the  technology”  as 
cited  in  the  fratricide  Boards  of  Inquiry  reports.  There  was  an  over  reliance  on  the 
outputs  of  automated  functions  with  a  resulting  decision  bias. 

•  Lack  of  Comprehension  of  the  Tactical  Situation — Patriot  system  operators  were 
trained  to  successfully  complete  routine  drills,  rather  than  to  actively  construct  a 
parallel  mental  model  of  an  unfolding  tactical  situation  and  possess  a  deeper 
understanding  of  the  operational  picture.  This  lack  of  situational  awareness 
becomes  apparent  when  scenarios  that  differ  substantially  from  those  trained  are 
confronted,  leading  the  operator  with  little  choice  but  to  depend  wholly  on 
automation  for  any  decisions.  The  Army  Board  of  Inquiry  stated,  “the  system  is 
too  lethal  to  be  placed  in  the  hands  of  crews  trained  to  such  a  limited  standard.” 

Another  factor  was  the  practice  of  assigning  relatively  inexperienced  personnel  to 
the  ECS  and  ICC.  This  practice  accentuated  the  problem  of  automation  bias.  The  ARL 
report  is  careful  to  point  out  that  this  problem  stemmed  not  from  the  ability  levels  of  the 
operators,  but  rather  from  a  composite  effect  of  decisions  made  by  developers,  engineers, 
trainers,  and  others  that  led  to  a  gradual  but  unchecked  shift  from  manual  control  to  a 
mode  of  supervisory  control.  This  change  was  not  reflected  in  training  practices  or 
personnel  assignment  practices. 

Observations  offered  in  the  ARL  report  suggest  a  number  of  factors  concerning  why 
the  MANPRINT  domains  (in  particular  personnel,  training,  and  human  factors 
engineering)  were  not  adequately  addressed  during  the  growth  of  the  Patriot  as  a  system. 
Patriot  grew  from  a  system  slightly  improved  from  its  predecessor,  the  1960s  era  Hawk 
missile  system,  to  a  complex  weapon  capable  of  anti-ballistic-missile  defense.  A  review 
of  MANPRINT  documents  and  test  reports  indicates  that  the  methods  of  MANPRINT 
assessment  also  did  not  progress  with  advances  in  technology,  instead  viewing  potential 
weaknesses  through  a  framework  of  manual  control.  However,  many  of  the  problems 
with  human  performance  started  to  show  up  in  earlier  test  results,  and  potential 
showstoppers  were  disregarded  or  overlooked.  Such  problems  were  often  deemed 
“isolated  training  issues”  that  could  be  shunted  for  correction  to  institutional  or  unit 
training.  But  the  emerging  supervisory  control  capacity  of  the  system  required  a  new 
front-end  analysis,  job  restructuring,  and  training  that  addressed  situational  analysis, 
problem-solving,  and  decision-making.  In  addition,  Patriot  system  operator  expertise  was 
not  sufficiently  developed. 

The  ARL  report  recognized  that  the  MANPRINT  community  was  not  solely 
responsible  for  failing  to  call  attention  to  the  serious  consequences  of  control  and  training 
issues.  The  entire  Patriot  team,  from  the  materiel  and  combat  developers,  to  the  U.S. 
Army  Training  and  Doctrine  Command  (TRADOC)  Systems  Manager  (now  called 
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TRADOC  Capability  Manager),  the  test  and  evaluation  community,  and  the  prime 
contractor  did  not  address  the  recognized  problem.  The  bottom  line  is  that  the 
organizational  inclination  is  for  the  larger  community  to  render  human  perfonnance  and 
training  issues  as  “inconvenient,  troublesome  problems  with  no  clean  (i.e.,  technology- 
oriented)  solution”  (Hawley  2007,  p.  10). 

The  human-performance  conditions  and  operational  context  that  impacted  the 
fratricides  led  to  two  actionable  items:  (1)  reexamine  the  roles  of  operators,  the  command 
and  control  structure,  and  the  concepts  of  automation  to  realize  the  human  supervisory 
control  task  and  (2)  examine  the  real  level  of  expertise  needed  to  operate  the  Patriot 
system  in  contemporary  operating  environments. 

2.  Defense  Science  Board  Report 

The  Defense  Science  Board  created  a  task  force  to  examine  the  lessons  from  the 
Patriot  systems  performance  in  Operation  Iraqi  Freedom  and  assess  whether  these  lessons 
could  be  incorporated  into  future  development  of  the  Patriot  or  follow-on  systems.  The 
Board  convened  between  August  2003  and  June  2004,  issuing  a  final  report,  which  is 
classified,  later  that  year.  In  the  unclassified  summary  report  (DSB  2005),  the  Board 
declared  that  the  operating  protocol  was  largely  automatic,  and  the  operators  were  trained 
to  trust  the  infonnation  generated  by  the  system’s  software.  The  proposed  solution  was 
“more  operator  involvement  and  control  in  the  functioning  of  the  Patriot  battery,  which 
will  necessitate  changes  in  software,  displays,  and  training”  (DSB  2005,  p.  2). 

3.  International  Training  of  Patriot  Operators 

Patriot  is  in  service  in  Egypt,  Germany,  Greece,  Israel,  Japan,  Kuwait,  the 
Netherlands,  Saudi  Arabia,  and  Taiwan.  The  training  for  Patriot  operators  preparing  for 
operational  assignments  is  done  differently  by  some  national  defense  forces.  In  Israel,  for 
example,  Patriot  operators  participate  in  4  months  of  initial  training,  rather  than  10 
weeks,  and  are  then  allowed  to  observe  in  the  ECS  under  supervision,  rather  than  serve 
immediately  as  an  operator  (Hawley,  Mares  and  Giammanco  2006).  They  remain  in  this 
supervised  position  for  up  to  1  year,  depending  on  their  individual  skill  progression. 
Upon  successful  skill  progression,  they  receive  2  months  of  higher  level  training  and  are 
then  considered  qualified  to  serve  as  a  tactical  control  assistant.  From  this  pool,  the  top 
students  are  selected  for  officer  training,  where  in  addition  to  officer  basic  training,  they 
undergo  a  4-month  air  defense  officer  course  and  then  serve  as  a  low-level  Tactical 
Control  Officer.  After  an  additional  year  of  experience  in  a  unit  enhanced  with  substantial 
simulation-based  training,  qualified  soldiers  participate  in  2  more  months  of  air  defense 
tactics  training  and  are  then  qualified  to  serve  as  high-level  Tactical  Control  Officers. 
Altogether,  more  than  30  months  of  training  combined  with  unit  experience  is  required 
for  the  needed  level  of  expertise  to  serve  as  a  Tactical  Control  Officer,  undergoing 
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rigorous,  performance-based  certification.  There  is  a  similar  story  with  the  training  in 
other  nations  (i.e.,  Germany  and  Denmark).  The  salient  point  is  that  these  militaries 
require  much  more  training  and  job  preparation  to  become  Patriot  operators  than  do  U.S. 
forces. 

4.  Terminal  High  Altitude  Air  Defense 

The  THAAD  is  a  separate  air  defense  missile  system  that  complements  the  lower- 
tier  Patriot  system  and  the  Navy’s  upper-tier  Aegis  ballistic  missile  defense  system.  It  is 
designed  to  destroy  short-range  and  medium-range  theater  ballistic  missile  threats. 
Although  originally  an  Army  program,  the  program  office  for  the  THAAD  is  part  of  the 
Ballistic  Missile  Defense  Systems  under  the  Missile  Defense  Agency.  The  first  THAAD 
firing  battery  was  activated  in  2008  and  received  all  hardware  and  components  for  full 
operation  in  early  2012.  The  interceptor  missile  has  a  range  of  about  124  miles.  Of 
concern  to  the  present  case  study  is  the  potential  for  the  lessons  from  Patriot  not  being 
applied  to  the  relatively  early  deployment  of  the  THAAD.  As  suggested  by  the  Defense 
Science  Board  (2005),  the  lessons  from  the  Patriot  are  useful  to  be  incorporated  into  other 
air  defense  systems. 

E.  Patriot  Summary  and  Case  Study  Implications 

The  implications  of  this  case  study  address  upgrading  training  for  a  system  as  its 
technical  capabilities  are  undergoing  an  upgrade.  The  training  that  may  have  been 
acceptable  for  an  early  operational  capability  is  not  necessarily  adequate  as  the  mission 
addresses  new  threats  and  the  system  takes  advantage  of  technological  advances.  In  the 
case  of  the  Patriot  missile  defense  system,  upgrades  over  a  period  of  more  than  20  years 
were  not  countered  with  enhancements  to  operator  training.  Just  as  advances  in 
electronics,  sensors,  and  propulsion  pave  the  way  for  a  more  capable  missile  defense,  so 
too  do  advances  in  training  technology,  the  learning  sciences,  and  the  engineering  of 
instruction  pave  the  way  for  a  better  trained  operational  force. 

This  case  study  has  demonstrated  the  impact  of  early  program  issues  with  systems 
training,  the  lack  of  training  development  with  technology  upgrades,  an  acceptance  of  the 
status  quo,  and  an  aftermath  of  consequences.  The  early  training  issues  began  surfacing 
even  with  the  baseline  system.  Research  on  the  operator’s  task  reflected  its  complex 
nature.  The  initial  job  task  analysis  resulted  in  an  almost  unwieldy  task  of  air  battle 
engagement  broken  into  more  than  100  subtasks  and  steps.  Further,  the  training  analysis 
did  not  appear  to  keep  up  with  the  system  upgrades.  The  training  of  routine  drills  with 
limited  scenarios  was  criticized  in  the  Board  of  Inquiry  report  after  the  Operation  Iraqi 
Freedom  fratricides;  the  report  called  for  training  that  would  improve  high-level 
judgment.  Unit  evaluations  supported  the  view  of  overall  preparedness  based  on 
satisfactory  performance  of  routine  battle  drills.  Also,  evidence  was  published  that 
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training  routines  for  Patriot  operators  did  not  generalize,  or  transfer,  to  more  complex, 
stressing  scenarios  with  conflicting  information. 

The  larger  implications  are  drawn  from  the  lessons  offered  in  the  Hawley  (2007) 
report.  The  first  is  that  if  a  flawed  system  concept  (i.e.,  training  approach)  is  not  corrected 
early  and  carried  into  development,  it  becomes  more  difficult  to  modify  and  achieve  the 
needed  level  of  performance,  in  this  case  human  performance.  It  was  not  the  case  that  the 
automation  of  certain  system  functions  could  override  the  impact  of  limited  training.  A 
deeper  understanding  of  the  operator-system  relationship  was  needed,  especially  as  the 
nature  of  this  relationship  changed  with  system  upgrades. 

The  second  lesson  is  that  training  issues  really  matter.  The  training  lacked 
substantial  rigor,  surprise  elements,  and  guided  practice.  There  were  early  attempts  at 
developing  such  training  approaches,  such  as  the  experimental  Intelligent  Conduct  of  Fire 
Trainer  developed  in  the  late  1980s  by  Bolt,  Beranek  &  Newman  Inc.  that  overlaid 
student  performance  and  expert  solutions  in  a  real-time  simulation,  with  feedback  from 
an  articulate  expert  in  the  fonn  of  computer  text  (Youngblut  1995).  Advanced 
development  of  such  intelligent  tutoring  systems  that  afforded  more  rigorous  training  was 
possible,  but  not  pursued.  The  third  lesson  from  this  report  is  that  testing  must  be  more 
comprehensive  and  rigorous.  In  the  case  of  testing  a  post-deployment  build,  limited  user 
test  for  the  Patriot  in  the  2006  time  frame,  the  impact  of  soldier  performance  was  a 
secondary  issue,  more  of  a  supportability  issue.  Human  performance  was  measured  on 
the  basis  of  utility  judgments  by  users,  which  are  subjective  and  of  limited  value  in 
addressing  performance  issues.  They  may  be  useful  in  a  fonnative  evaluation,  but  not  as 
indicators  of  the  ultimate  outcome  of  operator-system  performance.  Test  players  are  not 
adequately  trained  for  analysts  to  clearly  understand  the  issues  emerging  from  test  data. 
Was  it  a  system  or  operator  problem? 

The  program  manager  should  work  closely  with  the  training  community  to  develop 
options  for  individual  and  collective  training  for  operators,  maintainers,  and  support 
personnel  that  are  based  on  training  effectiveness  evaluations  to  maintain  skill 
proficiencies  and  reduce  individual  and  collective  training  costs.  The  program  manager  is 
responsible  for  developing  training  system  plans  to  maximize  the  use  of  new  learning 
techniques,  simulation  technology,  embedded  training  and  distributed  learning  (DoD 
Instruction  1322.26),  and  instrumentation  systems  that  provide  “anytime,  anyplace” 
training  and  reduce  the  demand  on  the  training  establishment. 

Another  source  of  guidance  comes  from  TRADOC  Pamphlet  525-8-2,  The  Army 
Learning  Concept  for  2015,  which  describes  the  transformation  of  the  soldier  training 
process  (TRADOC  2011).  Instead  of  traditional  lecture-based  classroom  training,  soldiers 
will  be  engaged  in  collaborative  learning  exercises;  receive  tailored,  blended  training;  and 
have  anytime,  anywhere  access  to  instructional  material.  Tedious  classroom  presentations 
will  be  replaced  by  engaging,  scenario-based  exercises  that  instructors  will  author  as 
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needed  to  represent  the  most  current  knowledge  from  theater.  The  ultimate  goal  of  these 
and  other  recommendations  is  to  provide  soldiers  with  relevant,  personalized  training  in  a 
technology-enabled  format  that  maximizes  training  benefits.  Whether  this  concept  is 
carried  into  the  early  plans  and  development  for  future  weapon  systems  remains  to  be 
seen. 

Finally,  how  well  an  operator  initially  learns  the  tasks  is  of  great  importance  to 
overall  system  performance.  Many  studies  indicate  that  the  single  most  important  factor 
affecting  skill  retention  is  the  degree  of  original  learning  (Arthur,  et  al.  1998).  If  the 
Patriot  operator  is  trained  well  from  the  start,  it  will  have  a  lasting  impact  on  the 
operational  effectiveness  of  the  Patriot  system. 

F.  Recommendations 

•  System  upgrades  require  training  upgrades;  improvements  in  systems  hardware 
and  software  must  be  accompanied  by  corresponding  training  enhancements  at  all 
levels. 

•  As  hardware  and  software  are  upgraded  in  the  system,  consider  similar  technical 
upgrades  in  training  systems. 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

•  Employ  training  lessons  learned  over  the  years  in  Patriot  to  the  THAAD  program. 
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3.  Embedded  Training  in  the 
Future  Combat  Systems 

— John  E.  Morrison 


A.  FCS  Background 

The  concept  for  the  FCS  program  arose  from  a  series  of  studies  conducted  in  the 
1990s  under  the  Army  After  Next  (AAN)  initiative  and  culminated  in  the  Army’s 
Transformation  Campaign  Plan,  which  was  described  by  then  Anny  Chief  of  Staff 
General  Eric  Shinseki  (2000).  AAN  studies  revealed  a  gap  in  U.S.  ground  force 
capabilities  based  on  lessons  learned  from  post-Cold  War  operations  and  performed 
analyses  of  then-current  Army  assets.  Despite  their  unparalleled  lethality,  survivability, 
and  staying  power,  U.S.  heavy  forces  lacked  strategic  responsiveness  and  deployability 
and  had  an  unacceptably  large  logistical  footprint.  In  contrast,  the  Army’s  light  forces 
were  responsive  and  deployable  but  were  deficient  in  terms  of  lethality  and  survivability 
and  lacked  staying  power.  To  fill  this  capability  gap,  Army  planners  proposed  to 
transfonn  the  Army  into  a  lethal  yet  strategically  responsive  fighting  force.  This 
transformation  process  entailed  reorganizing  the  combat  force  into  smaller  and  more 
standardized  units  equipped  with  the  light,  highly  mobile,  and  lethal  combat  weapon 
systems  that  would  be  wirelessly  linked  to  enable  network-centric  operations.  These 
proposed  weapon  systems  were  designed  and  developed  under  the  collective  umbrella  of 
the  FCS  program. 

As  depicted  in  Figure  3-1,  the  FCS  was  conceived  as  a  family  of  weapon  systems, 
comprising  eight  types  of  manned  vehicles:  Mounted  Combat  System,  Infantry  Carrier, 
Non  Line-of-Sight  Mortar,  Non  Line-of-Sight  Cannon,  Reconnaissance  and  Surveillance, 
Command  and  Control,  Medical,  and  Recovery  and  Maintenance.  Although  these  FCS 
vehicle  types  differed  in  functions  and  capabilities,  they  shared  a  common  chassis  and 
engine  to  reduce  the  logistical  burden  of  maintaining  the  combat  force.  Furthermore,  all 
FCS  vehicles  were  originally  designed  to  be  limited  in  size  and  weight  to  allow  them  to 
be  transported  by  the  Air  Force’s  large  fleet  of  C-130  aircraft.  Perhaps  the  most 
innovative  aspect  of  the  FCS  was  that  it  connected  the  disparate  vehicles  and  systems  to  a 
common  network.  This  network  was  designed  to  allow  the  family  of  FCS  vehicles  and 
systems  to  act  as  a  coherent  whole  and  to  conduct  network-centric  military  operations. 


The  FCS  Program  also  included  unmanned  systems,  but  they  were  not  considered  relevant  to  the 
embedded  training  capability  and  are  not  considered  here. 
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One  of  the  key  services  provided  by  the  FCS  network  was  the  embedded  capability  to 
train  anywhere  and  anytime  in  live,  virtual,  and  constructive  simulation  environments. 


■  Manned  Systems 


■  Unmanned  Air  Platforms 


Command  and 
Control  Vehicle 


Class  I 


Class  III 


Class  IV 


Infantry  Carrier 
Vehicle 


Unattended 

Munitions 

•  NLOS  LS 

•  Intelligent 
Munitions 


Unattended  l 
Ground  Sensors 


Mounted  “ 
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Recon  and 
Surveillance 


Unmanned  Ground  Vehicles 
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Armed 

Robotic  Vehicle 
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Medical  Treatment 
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FCS  Maintenance 
and  Recovery  Vehicle 


Figure  3-1.  FCS  conceptual  diagram  as  of  September  2004. 

B.  FCS  System  Overview 

Many  of  the  core  warfighting  concepts  and  technologies  in  the  FCS  program  were 
first  raised  in  the  AAN  program  in  the  1990s,  including  the  capability  to  provide 
embedded  training.  As  described  in  a  1996  Defense  Advanced  Research  Projects  Agency 
(DARPA)  paper,  the  AAN  would  require  new  ground  combat  platforms  that  embed 
training  capabilities  into  onboard  command,  control,  communications,  computers,  and 
intelligence  systems  (Gully  et  al.  1996).  These  embedded  training  capabilities  would 
include  tactical  engagement  simulation  (TES)  for  live  simulation  and  after  action  review 
systems  for  enhancing  performance  feedback  from  training  exercises  and  combat 
operations. 

The  emphasis  on  embedded  training  was  carried  through  to  conceptual  phase  of 
FCS  program.  In  May  2000,  DARPA  awarded  contracts  to  four  industry  teams  to  study 
technologies,  missions,  and  tasks  required  to  conduct  combat  operations  anticipated  for 
the  FCS.  According  to  the  original  solicitation  in  January  2000,  the  team  proposals  must 
include  the  following: 

A  training  system/capability  for  the  soldier  as  well  as  maintenance.  The 
system  training  concepts  are  encouraged  to  be  imbedded,  minimizing 
additional  equipment  and  manpower  requirements.  In  addition,  the  cost  of 
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training  should  be  addressed  as  well  as  the  advantages/disadvantages  of 
virtual  vs.  actual  operation  of  the  equipment  during  training  operations. 
(DARPA  2000,  para  4.5. i) 

In  April  2003,  TRADOC  released  the  FCS  Operational  Requirements  Document 
(ORD).  The  FCS  ORD  identified  seven  Key  Performance  Parameters  (KPPs),  which  are 
measurable  performance  objectives  that  FCS  must  attain  to  enable  its  mission  and 
operational  goals.  Training  was  called  out  as  a  separate  KPP  for  FCS,  which  was 
unprecedented  for  an  MDAP  weapon  system  at  that  time.  The  FCS  Training  KKP  was 
defined  as  follows: 

The  FCS  FoS  [family  of  systems]  must  have  an  embedded  individual  and 
collective  training  capability  that  supports  live,  virtual,  and  constructive 
training  environments.  (Department  of  the  Army  2003) 

In  the  FCS  ORD,  critical  technologies  were  identified  for  each  FCS  KPP.  Two 
critical  technologies  were  deemed  necessary  to  meet  the  Training  KPP  objective: 

•  Computer  Generated  Forces  (CGF),  which  provides  the  virtual  and  constructive 
simulation  training  environment  for  embedded  training. 

•  Tactical  Engagement  Simulation  System  (TESS),  which  enables  valid  simulations 
of  enemy  and  friendly  weapons  effects  required  for  live  training. 

Note  that  the  FCS  embedded  training  system  is  not  an  appended  embedded  training 
system  or  add-on  feature  of  the  weapon  system.  Indeed,  one  of  the  first  principles  for 
design  of  the  FCS  embedded  training  system  is  that  it  be  developed  in  parallel  with  and 
fully  integrated  into  the  operational  system  design  (Shiflett  2009).  As  shown  in  Figure 
3-2,  the  FCS  embedded  training  system  was  integrated  with  four  of  the  five  architectural 
layers  of  the  FCS  network:  transport  (telecommunications),  services  (middleware  that 
provides  interoperability  within  and  across  platforms),  applications  (software  packages 
for  10  battle  command  functions),  and  sensors/platforms  (distributed  and  networked  array 
of  multi-spectral  sensors).  This  level  of  integration  ensures  that  no  additional  systems  or 
capabilities  are  required  to  immerse  soldiers  in  realistic  embedded  training  scenario. 


3 

The  other  six  KPPs  were  Joint  Interoperability,  Networked  Battle  Command,  Networked  Lethality, 
Transportability,  Sustainability/Reliability,  and  Survivability. 
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Integrated  Warfighting  Platforms:  Participating  in  live, 
virtual,  constructive  and  multi-mode  training  exercise  as  a 
trainee  platform  or  supporting  platform 
Sensing  Systems:  Exchanging  live  and  simulated 
sensor  data  through  system  of  system  interfaces 

Battle  Command  Applications:  Integrated  Embedded 
Training  leveraging  Battle  Command  software  and  training 
common  components;  management  of  networks  in  live, 
virtual,  and  constructive  training;  stimulating  Battle 
Command  through  System  of  System  Interfaces 

Service  Layer:  Routing  training  data  over  FCS  network; 
separating  operations  and  training  data,  training  data 
access,  logging  data 

Transport  Systems:  Interface  to  live  training  facilities, 
distribute  collective  training  data  over  wire  and  wireless 
networks,  training  data  reach,  complementary  program 
interoperability 

Ref:  SSDD,  section  (4.2.9)  ACE#  D786-10990-1  Rev.  K 

From  Shiflett  (2009). 

Figure  3-2.  Diagram  showing  integration  of  embedded  training  functionality  into 

operational  system  architecture. 

C.  FCS  Technology  Development 

FCS  technologies,  including  its  embedded  training  system,  were  developed  over  a 
10-year  period  (from  2000  to  2009).  Figure  3-3  is  a  time  line  of  significant  events  that 
relate  to  the  FCS  system  as  a  whole  and  to  the  embedded  training  system  in  particular. 
Phase  1  (Concept  Development)  of  FCS  began  in  2000  with  the  DARPA  solicitation  and 
lasted  approximately  2  years.  In  March  2002,  the  Army  chose  the  team  of  Boeing  and 
Science  Applications  International  Corporation  to  serve  as  lead  systems  integrator 
overseeing  key  aspects  of  the  FCS  development  process.  At  Milestone  A  in  May  2002, 
the  FCS  Program  moved  to  Phase  2  (Technology  Development).  One  year  later  (May 
2003)  at  Milestone  B,  the  Defense  Acquisition  Board  approved  FCS  to  move  to  Phase  3 
(System  Development  and  Demonstration).  In  August  2004,  the  lead  systems  integrator 
awarded  21  companies  contracts  to  develop  FCS  vehicles,  platforms,  and 
hardware/software  (Tiboni  2004).  The  product-development  phase  lasted  for  about  5 
years  and  had  been  planned  to  lead  to  a  Go/No  Go  Milestone  C  decision  of  whether  the 
program  should  enter  into  the  production  phase.  However,  in  June  2009,  Secretary  of 
Defense  Robert  Gates  issued  an  acquisition  decision  memorandum  canceling  the  FCS 
program.  In  explaining  his  decision,  Secretary  Gates  said  that  there  were  “significant 
unanswered  questions  concerning  the  FCS  vehicle  design  strategy... we  must  have  more 
confidence  in  the  program  strategy,  requirements,  and  maturity  of  the  technologies  before 
proceeding  further”  (Gates  2009).  However,  as  discussed  below,  the  unanswered 
questions  that  led  to  cancellation  of  the  FCS  program  did  not  pertain  to  its  training 
system  or  related  technologies. 
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Figure  3-3.  Technology-development  events  relating  to  the  FCS  as  a  whole  (above  the  time 
line)  and  to  the  embedded  training  system  in  particular  (below  the  time  line). 


Concerns  about  maturity  of  FCS  technologies  were  expressed  throughout  the 
program  design  and  development.  As  early  as  the  initiation  of  the  System  Development 
and  Demonstration  (SDD)  phase  in  2003,  it  was  noted  that  many  of  the  proposed  FCS 
technologies  were  insufficiently  mature  to  start  vehicle  system  development  (GAO 
2003).  Maturity  of  FCS  critical  technologies  was  tracked  using  the  Technology  Readiness 
Level  (TRL)  construct  pioneered  by  the  National  Aeronautics  and  Space  Administration 
and  frequently  used  in  DoD  acquisition  programs  incorporating  new  technologies. 
Appendix  B  describes  Technology  Readiness  Levels  1  through  9  as  the  GAO  has  applied 
the  three  levels  to  the  FCS  program.  The  GAO  has  consistently  maintained  that  it  is  an 
industry  best  practice  and  DoD  policy  preference  to  mature  technologies  to  at  least  TRL  7 
before  the  start  of  product  development — that  is,  before  the  start  of  SDD  phase,  which 
occurred  in  May  2003.  TRL  7  requires  that  the  technology  be  in  prototype  form  and 
possess  the  form,  fit,  and  function  of  the  finished  product  as  demonstrated  in  a  realistic 
environment.  For  the  FCS  program,  the  Army  had  a  substantially  more  lenient  maturity 
goal:  All  critical  technologies  for  FCS  had  to  be  at  TRL  6  before  going  into  the 
production  phase  that  had  been  scheduled  for  May  2009. 

Table  3-1  tracks  the  actual  maturation  of  the  critical  technologies  from  the  start  of 
the  FCS  SDD  phase  in  2003  until  just  before  program  cancelation  in  January  2009.  The 
interpretation  of  progress  made  over  that  time  differs  depending  on  the  maturation  goal 
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adopted.  According  to  the  GAO  goal,  the  FCS  program  was  not  ready  to  enter  the  SDD 
phase  in  2003  and  had  made  little  progress  since  then,  showing  only  7%  of  technologies 
fully  mature  in  2009.  By  the  Army  goal,  nearly  all  critical  technologies  (91%)  had 
matured  to  TRL  6  by  January  2009,  and  the  remaining  four  critical  technologies  at  TRL  5 
were  scheduled  for  TRL  6  demonstrations  before  the  system-of-systems  preliminary 
design  review  (Milestone  C)  that  was  scheduled  for  May  2009.  However,  unlike  the 
previous  three  assessments,  not  all  critical  technologies  listed  in  the  January  2009 
assessment  had  been  validated  by  an  independent  review  team.  Previous  independent 
review  teams  downgraded  initial  assessments  provided  by  the  LST,  and  questions 
remained  about  maturity  of  three  critical  technologies  in  particular:  (1)  the  Joint  Tactical 
Radio  System  ground  mobile  radio,  (2)  Mobile  Ad-hoc  Networking  Protocols,  and  (3) 
Wideband  Networking  Waveforms.  So  even  using  the  more  lenient  standard,  there  was 
some  concern  that  all  FCS  technologies  would  not  be  sufficiently  mature  to  enter  the 
production  phase  of  development. 


Table  3-1.  Technical  Readiness  Levels  for  FCS  critical  technologies. 


Technical  Readiness 
Level 

Program  Start 
2003 

August  2006 

July  2007 

January  2009 

No. 

(%) 

No. 

(%) 

No. 

(%) 

No. 

(%) 

TRL  >  7 

0 

(0) 

1 

(2) 

2 

(5) 

3 

(7) 

TRL  >  6 

10 

(19) 

35 

(76) 

32 

(73) 

40 

(91) 

TRL  <  5 

42 

(81) 

11 

(24) 

12 

(27) 

4 

(9) 

Total  critical 

52 

46 

44 

44 

technologies 

Note:  Data  from  GAO  (2009). 


Table  3-2  focuses  on  the  two  critical  technologies  associated  with  the  embedded 
training  system.  This  table  indicates  that  the  CGF  critical  technology  was  relatively 
mature  soon  after  the  start  of  SDD.  The  independent  review  team  justified  the  TRL  6 
assessment  of  computer  generated  forces  as  follows: 

OneSAF  CGF  [computer  generated  forces]  has  been  in  continuous  use  for 
at  least  a  decade.  It  is  being  upgraded/expanded  to  provide  a  greater 
variety  of  the  conflict  interactions,  but  urban  dismounts  being  tested  in 
FCS-C2,  stabilization  and  peacekeeping  are  not  yet  included.  (Delaney  et 
al.  2004) 
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Table  3-2.  Technological  Readiness  Levels  for  embedded  training  critical  technologies. 


September 

April 

August 

2006c 

July 

Critical  Technologies 

2004a 

2005b 

2007d 

Computer  Generated  Forces 

6 

6 

6 

6 

Tactical  Engagement  Simulation  System 

4 

4 

5 

6 

a  Data  from  Independent  Review  of  Technology  Maturity  Assessment  for  Future  Combat  Systems  (Delaney, 


et  al.  2004) 

b  Data  from  GAO  (2006). 
c  Data  from  GAO  (2007). 
d  Data  from  GAO  (2008). 

In  contrast  to  CGF  technology,  the  TESS  started  at  TRL  4  but  achieved  TRL  6  by 
the  July  2007  assessment.  The  initial  problem  was  that  the  TESS  architecture  had  not 
been  developed  in  any  detail  and  it  was  not  clear  how  it  would  interface  with  other  FCS 
systems  (e.g.,  the  Joint  Tactical  Radio  System)  and  external  live- simulation  architecture 
(e.g.,  the  Common  Training  Instrumental  Architecture  used  at  Army  Combat  Training 
Centers).  Most  of  these  issues  were  resolved  with  development  and  demonstration  of  the 
OneTESS  architecture  that  successfully  integrated  OneSAF  and  live  training  simulation 
architectures.  Thus,  while  some  of  the  FCS  critical  technologies  lagged  in  development, 
the  technologies  related  to  embedded  training  (CGF  and  TESS)  were  relatively  mature 
well  before  the  planned  start  of  production  in  2009. 

D.  FCS  Training  System  Issues 

Even  though  the  FCS  program  was  canceled,  the  embedded  training  system  is 
worthy  of  study  if  for  no  other  reason  than  it  was  the  training  system  in  the  most 
ambitious  Army  modernization  program  in  history.  An  argument  can  also  be  made  that 
future  net-centric-capable  systems  will  provide  enhanced  opportunities  to  maximize  the 
potential  of  embedded  training  systems.  Furthermore,  as  discussed  below,  there  are  three 
specific  aspects  of  the  embedded  training  system  within  the  FCS  that  argue  for  including 
it  as  a  case  study  as  well. 

1.  Embedded  Training  as  Integral  Capability 

The  embedded  training  system  was  considered  an  essential  capability  for  the  entire 
FCS  family  of  systems.  The  perceived  capability  gaps  in  deployability  and 
responsiveness  gave  rise  to  the  requirement  to  train  anytime  and  anywhere  for  a  variety 
of  potential  missions.  Given  these  requirements,  it  would  be  difficult  to  imagine,  much 
less  design,  an  FCS  family  of  systems  without  an  embedded  training  capability. 
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2.  Embedded  Training  as  Key  Performance  Parameter 

One  of  the  unique  aspects  of  the  FCS  program  was  that  it  elevated  training  to  the 
level  of  a  system  KPP.  This  ensured  that  training  remained  a  high-visibility  concern 
throughout  the  history  of  FCS  program.  And  as  the  current  Joint  Capabilities  Integration 
Development  System  manual  notes,  having  training  as  a  system  KPP  provides  a  hedge 
against  the  historic  problem  of  trading  away  training  resources  to  supplement  the 
increased  cost  of  the  parent  system  (DAU  2012). 

Another  perhaps  less  obvious  implication  of  raising  training  to  KPP  status  is  that  it 
requires  the  system  developer  to  monitor  critical  training  technologies.  For  the  FCS,  that 
amounted  to  increased  concern  about  the  maturity  of  CGF  and  TESS  technology.  In  fact, 
that  led  to  the  early  identification  and  eventual  resolution  of  problems  of  integrating 
TESS  with  systems  within  the  FCS  platfonns  and  with  architectures  of  live  training 
instrumentation  systems  at  major  combat  training  centers.  Had  this  technology  not  been 
monitored,  the  TESS  issue  could  have  languished  from  lack  of  attention  and  not  been 
addressed  appropriately. 

3.  Embedded  Training  as  Technology  Spin  Out 

When  the  FCS  program  was  canceled,  some  of  the  FCS  technologies  were 
transitioned  to  the  Brigade  Combat  Team  Modernization  (BCTM)  program.  Not  included 
in  this  transition  was  the  Manned  Ground  Vehicle  (MGV),  which  was  the  primary 
platform  for  the  embedded  training  system.  As  a  result  of  the  MGV  cancellation,  many  of 
the  features  of  the  embedded  training  system  were  lost.  However,  some  aspects  of  the 
FCS  embedded  training  were  incorporated  into  the  BCTM  Embedded  Training  Software 
(BETS).  Perhaps  the  most  important  aspect  transitioned  from  the  FCS  program  was  the 
design  principle  that  BETS  be  integrated  into  the  operational  BCTM  Network  and 
Mission  Command  Software.  The  final  version  of  the  BETS  was  demonstrated  and 
delivered  to  the  BCTM  program  in  May  2011  (Shiflett  2011).  That  the  FCS  embedded 
training  system  was  successfully  spun  off  to  the  BCTM  program  attests  to  the  maturity 
and  viability  of  its  design  principles. 

E.  FCS  Summary  and  Case  Study  Implications 

The  FCS  program  embedded  training  system  is  considered  a  good  model  of  training 
development  for  MDAPs.  Discussed  below  are  positive  aspects  of  the  FCS  embedded 
training  systems  design  and  development  that  apply  to  MDAPs  in  general  and  not  just  to 
the  FCS  in  particular. 

1.  Stress  Training  from  Outset 

Even  in  the  earliest  conceptual  discussions  of  the  proto-FCS  system  in  the  AAN 
studies,  embedded  training  was  considered  an  essential  capability  of  the  proposed 
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weapon  system.  As  the  FCS  moved  into  the  initial  conceptual  phase,  studies  and  analyses 
were  required  to  assume  that  embedded  training  was  a  fundamental  system  capability. 
Then  as  the  FCS  program  moved  to  the  Technology  Development  (TD)  phase,  training 
was  defined  as  a  KPP.  This  stress  on  training  enabled  the  topic  to  emerge  and  be 
maintained  as  a  principal  consideration  throughout  the  design  development  of  the  weapon 
system. 

The  early  incorporation  of  training  into  system  design  and  development  is  consistent 
with  current  guidance  on  the  Joint  Capability  Integration  Development  System.  The  Joint 
Capabilities  Integration  Development  System  Manual  (dated  19  January  2012)  maintains 
that  training  requirements  should  be  addressed  in  parallel  with  planning  and  materiel 
development  (DAU  2012).  Specifically,  this  manual  asserts  that  training  KPPs  are 
required  for  all  MDAPs.  Although  formal  KPPs  are  not  required  for  Milestone  A  (entry 
into  TD  phase),  the  guide  requires  that  an  initial/draft  training  plan  be  developed  during 
the  Materiel  Solution  Analysis  phase  prior  to  Milestone  A.  The  FCS  program  essentially 
performed  these  studies  in  the  pre-TD  phase  of  conceptual  development  from  2000  to 
2003. 

2.  Integrate  Embedded  Training  with  Weapon  System 

One  of  the  more  remarkable  aspects  of  the  FCS  embedded  training  system  was  the 
degree  to  which  the  embedded  training  system  was  integrated  into  the  FCS  operational 
hardware  and  software.  Such  integration  provides  different  types  of  benefits.  First, 
integration  of  training  with  operational  system  benefits  training  by  (1)  maximizing 
training  fidelity  in  that  it  allows  individuals  and  units  to  train  as  they  fight  and  (2) 
enabling  the  ability  of  units  to  train  anytime  and  anywhere  the  operational  systems  are 
available — including  during  deployments.  Second,  integration  benefits  overall  system 
development  by  reducing  needless  duplication  in  software  development  and  by  avoiding 
increases  in  size,  weight,  and  power  requirements  associated  with  appended  training 
systems. 

Another  important,  albeit  nonobvious,  benefit  of  the  integration  of  training  and 
operational  systems  is  that  it  reduces  baseline  divergence  of  training  systems  and 
operational  software  (TRADOC  2006).  As  illustrated  in  previous  cases,  such  divergence 
is  common  if  the  training  system  is  developed  as  a  separate  component  from  the 
operational  system.  In  contrast,  integrating  training  and  operational  system  provides  an 
inherent  constraint  for  ensuring  that  training  upgrades  are  synchronized  with  system 
upgrades. 

3.  Monitor  Maturity  of  Critical  Training  Technologies 

Simply  stressing  training  in  materiel  development  is  not  enough.  Training  system 
developers  need  to  provide  due  diligence  by  closely  monitoring  the  maturity  of 
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technologies  required  to  implement  training  effectively.  For  FCS,  the  TESS  technology 
was  identified  as  potentially  problematic.  However,  early  identification  of  problems 
enabled  developers  to  find  appropriate  solutions.  Similarly,  newer  training  systems  may 
be  dependent  upon  other  advanced  technologies,  such  as  models  of  text  understanding, 
methods  for  measuring  eye  fixations  and  movements,  and  brain  activity  monitoring.  If 
such  technologies  are  deemed  “critical”  for  the  training  system,  then  developers  must 
monitor  their  maturity  closely  to  ensure  that  the  technologies  will  actually  work  as 
intended  in  the  operational  environment. 

F.  Recommendations 

•  Stress  training  from  outset — All  MDAPs  (including  rapid  track  acquisitions) 
should  address  training  as  an  integral  part  of  the  acquisition  program. 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

•  Integrate  embedded  training  with  MDAP  weapons  systems. 

•  Monitor  technical  maturity  of  critical  training  technologies. 

•  Provide  training  with  the  fielded  system  as  a  package  to  realize  full  capability  of 
new  systems. 
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4.  Mine  Resistant  Ambush  Protected  Vehicle 

— Anthony  Ciavarelli 


A.  Mine  Resistant  Ambush  Protected  Vehicle  Background 

1.  Mine  Resistant  Ambush  Protected  Vehicle  Acquisition  Historical  Summary 

A  GAO  study  stated  that  75%  of  all  casualties  in  Iraq  and  Afghanistan  as  of  July 
2008  were  attributed  to  improvised  explosive  devices  (IEDs).  To  reduce  this  casualty 
risk,  the  DoD  initiated  the  Mine  Resistant  Ambush  Protected  Vehicle  (MRAP)  Rapid 
Acquisition  Program,  which  sought  to  develop  better  armored  vehicles  (GAO  2009). 

The  DoD  MRAP  acquisition  program  began  in  2007  and  used  a  “tailored”  rapid 
acquisition  process  to  rapidly  acquire  and  field  protective  vehicles.  The  essential 
elements  of  a  rapid  acquisition  process  were  (1)  the  use  of  proven  technologies  and 
commercially  available  products;  (2)  the  establishment  of  minimal  operational 
requirements;  and  (3)  the  use  of  a  concurrent  development,  test,  and  evaluation 
framework.  DoD  made  the  acquisition  of  MRAPs  its  highest  priority  and  maintained 
direct  control  of  system  integration  (Feickert  2008).  Figure  4-1  shows  an  RG  33  MRAP. 


Source:  http://www.military-today.com/apc/rq33  mrap.jpq. 

Figure  4-1.  RG  33  MRAP. 

The  2008  Defense  budget  included  the  MRAP  funding  in  excess  of  $8  billion  for 
7,700  vehicles  across  the  services,  and  after  massive  production  in  2009,  the  cost  rose  to 
$22.7  billion  for  14,000  MRAPs.  Schedule  and  cost  performance  were  deemed  to  be  very 
good  overall. 
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B.  MRAP  System  Overview 

Following  rapid  production  and  deployment  of  MRAP  vehicles  (2007-2008),  it 
became  clear  that  sustaining  a  fleet  of  about  15,000  vehicles  “could  pose  significant 
challenges”  (Feickert  2008,  GAO  2008).  The  exact  vehicle  type  and  inventory  varied 
greatly  by  military  service,  and  they  also  varied  by  the  type  of  mission  and  theater  of 
operations;  but  all  vehicles  were  designed  in  a  similar  fashion — tall  and  heavy.  The 
vehicle  designs  gave  ample  protection,  in  most  cases,  but  at  a  price  of  increased  vehicular 
accidents — primarily  vehicle  rollovers. 

1.  Vehicle  Safety  Issues 

There  were  over  60  MRAP -related  mishaps  between  November  and  June  2008;  over 
one-half  of  these  were  rollovers  (USMC  2008).  The  Marine  Corps  reported  that  the  cost 
of  MRAP  mishaps  occurring  from  2008  to  2010  was  360  lost  work  days  and  $3,462,221 
in  material  losses  (HQ  USMC  2010).  HQ  USMC  also  identified  the  major  cause  of  most 
MRAP  accidents  as  follows:  “When  reported,  LACK  OF  TRAINING  is  identified  in  the 
narrative  for  class  A/B  (the  most  serious)  mishaps  60%  of  the  time.” 

The  death  of  three  U.S.  Army  Special  Forces  (Green  Berets)  in  Afghanistan  on  29 
June  2008,  which  occurred  when  their  RG-31  MRAP  rolled  into  a  river,  raised  awareness 
of  the  high  risk  of  rollovers  in  the  MRAP  fleet.  As  a  result,  DoD  began  to  study 
alternatives  for  mitigating  rollover  mishaps,  including  vehicle  redesign  and  additional 
driver  training  to  help  reduce  rollover  risk  (Feickert  2008).  Figure  4-2  shows  the  mishap 
statistics  reported  in  one  study  that  raised  awareness  of  the  safety  of  MRAP  operations  in 
Iraq  and  Afghanistan  (Rice  2008). 

As  shown  in  Figure  4-2,  the  majority  of  MRAP  mishaps  were  rollovers.  Nearly  75% 
of  the  rollovers  occurred  outside  urban  areas — typically  when  driving  on  soft  road 
shoulders,  riverbanks,  and  berms,  and  over  poorly  constructed  bridges.  In  most  cases, 
drivers  were  unaware  of  the  hazard,  such  as  poorly  constructed  bridges  and  berms.  When 
encountering  an  imminent  rollover,  drivers  typically  over-corrected  steering  because  they 
were  not  trained  to  initiate  proper  counter-maneuvers.  Following  a  rollover,  some 
personnel  perished  because  they  could  not  exit  the  vehicle  in  time  to  avoid  a  resulting  fire 
or  drowning  in  the  river  or  canal  water. 
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UNCLASS 


MRAP  Mishaps  -  1  Nov  07-20  Sep  08 

•  122  mishaps  (recorded  from  multiple  sources*) 

•  63  (51%)  rollovers/tip-overs 

-  39  -  Fall  initiated  :  ledge,  slope,  ground  surface  collapsed 

-  14  -  Maneuver  initiated:  swerving  maneuver  on  flat  ground  or  terrain 

-  3  -  Impact  Initiated:  hitting  object 

-  7  -  Unknown  cause 

•  7  fatalities  from  rollovers 

•  66  injuries  from  rollovers 

•  59  other  mishap  events  include: 


Event 

Number 

Traffic  Accident  with  other  vehicles 

12 

Personal  Injury  -  Falling  off/tripping 

6 

Personal  Injury  -  Crushing  (doors,  hatches.  Rhino,  turret) 

16 

Power  Line  Related 

10 

Fire 

4 

Other,  i.e.  hit  objects,  failed  to  set  parking  brake,  etc. 

11 

*  CENTCOM  SIGACTS,  Unit  Safety  Gram/Red-Hash,  Safety  Centers.  Data  includes  both  OiF  and  0EF. 

UNCLASS 


Source:  Slide  from  presentation  by  Rice  (2008) 

Figure  4-2.  Summary  of  MRAP  mishaps  in  FY2008. 


Figure  4-3,  taken  from  Rice  and  Rodriguez-Johnson  (2009),  compares  several 
vehicle  types  by  average  monthly  rollovers  per  1,000  fielded  vehicles.  The  figure  shows 
that  the  standard  high  mobility  multipurpose  wheeled  vehicle  (FIMMWV)  (unarmored) 
vehicle  was  also  subject  to  rollovers,  but  such  rollovers  were  mainly  due  to  excess  speed 
and  turning  too  severely  and  fast,  not  due  to  vehicle  design.  Evidence  from  anecdotal 
input  (discussed  later)  did  show  a  marked  increase  in  rollovers  for  the  Armadillo  (annor 
upgraded)  FIMMWV.  Figure  4-3  shows  that  the  MRAP-type  vehicles  had  a  greater 
rollover  rate  than  the  standard  HMMWV.  The  Stryker  (a  large,  heavy  MRAP-style 
vehicle)  had  about  four  times  the  standard  HMMWV  rollover  rate,  and  the  “generic” 
MRAP  had  about  twice  the  rollover  rate  of  the  standard  HMMWV. 
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Source:  Graph  from  Rice  and  Rodriguez-Johnson  (2009). 

Figure  4-3.  MRAP  Mishaps,  November  1,  2007 — August  31,  2009,  by  type. 

In  an  AP  story,  Lardner  (2008)  described  the  first  fatal  MRAP  rollover: 

A  crew  of  six  Soldiers  from  the  101st  Airborne  was  traveling  over  an 
irrigation  canal  in  a  Caiman  9-foot  tall,  19  ton  MRAP  when  the  driver 
turned  too  quickly  and  the  rear  tires  sank  in  the  dirt  shoulder  which 
collapsed  under  the  vehicle.  The  driver  tried  to  maneuver  out  of  the  rut  by 
accelerating  and  turning  quickly  but  the  vehicle  flipped  over  on  its  side 
and  rolled  down  an  embankment  into  the  canal.  It  sank  into  10  feet  of 
water  causing  the  power  to  shut  down,  leaving  the  crew  in  darkness.  Some 
crew  members  were  fortunate  to  find  an  air  pocket  and  were  later  rescued. 

But  two  crew  members  died  by  drowning. 

But  it  was  the  aforementioned  death  of  three  U.S.  Army  Special  Forces  (Green  Berets)  in 
an  RG-3 1  MRAP  that  dramatically  raised  awareness  of  the  high  risk  of  MRAP  rollovers 
(Feickert  2008). 


2.  U.S.  Marine  Corps  Operator  Anecdotal  Report 

Some  key  highlights  of  an  anecdotal  experience  depicting  the  impact  of  simulation 
training  for  the  armored  HMMWV  (Armadillo  version)  are  presented  below.  The  U.S. 
Marine  (Yates  2012)  describing  the  sequence  of  events  is  a  graduate  of  the  MOVES 
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Institute  Naval  Postgraduate  School  and  served  as  the  simulation  officer  for  the  Operator 
Driving  Simulator  (ODS). 

As  the  IED  became  the  primary  lethal  threat  to  troops  in  Iraq  we  rushed  to 
up-annor  our  vehicles.  The  “Armadillo”  annor  kit  that  was  applied  to  the 
7-ton  MTVR  [Medium  Tactical  Vehicle  Replacement]  truck  was  a  huge 
improvement  in  survivability  for  troops  riding  in  the  back.  Until  the  first 
MRAPs  were  fielded  the  MTVRs  with  the  Armadillo  kit  were  the  most 
survivable  wheeled  vehicles  in  Iraq  from  an  IED  blast.  However,  the 
accident  rates  began  to  skyrocket  when  trucks  were  upgraded  with  the 
Armadillo.  In  April  2006  the  first  two  Armadillo  equipped  MTVRs 
arrived  at  MCAGCC  [Marine  Corps  Air  Ground  Combat  Center]  for  use 
by  Marines  in  the  Mojave  Viper  Exercise.  Within  a  week  of  the  first  two 
trucks  being  delivered  the  driver  of  one  truck  rear-ended  the  truck  in  front 
of  him  because  he  was  following  too  closely  and  the  brakes  would  not 
stop  the  vehicle  before  it  collided.  The  damage  to  one  vehicle  was  so 
severe  that  it  had  to  be  returned  to  the  Oshkosh  factory  for  a  repair  of  a 
crack  in  the  chassis  (>$100k).  Over  the  next  six  weeks  at  MCAGCC  there 
were  an  average  of  two  operator  error  vehicle  mishaps  with  Armadillo 
equipped  MTVRs  every  week  and  many  involved  injury  to  Marines. 
Armadillos  did  not  accelerate,  brake,  or  corner  like  a  regular  MTVR.  Yet, 
in  the  hurry  to  field  the  more  survivable  Armadillo  package  there  was  no 
driver  training  component  developed  for  the  modified  vehicle.  Marines 
never  saw  or  got  a  chance  to  drive  an  Armadillo  equipped  truck  before 
deploying  to  Iraq  because  they  were  being  shipped  directly  to  Iraq  as  soon 
as  they  rolled  off  the  assembly  line  at  Oshkosh... 

Following  the  introduction  of  hands-on  Driver  Simulator  training,  a  remarkable 
improvement  was  observed: 

In  the  words  of  the  Operations  Officer  from  the  Mojave  Viper  Support 
Detachment,  “the  impact  of  the  ODS  trainer  was  immediate  and 
dramatic .”  From  the  day  that  ODS  training  began  in  May  2006  through 
the  end  of  that  calendar  year  there  was  not  another  mishap  involving  an 
Armadillo  variant  MTVR.  The  accident  rate  had  dropped  from  two  per 
week  to  zero. 

The  evidence  from  mishap  statistics,  this  Marine’s  experience,  and  the  experience  of 
others  as  reported  by  U.S.  Marine  Corps  safety  specialists  (Rice  2008,  USMC  2008, 
USMC  2010)  pointed  to  the  need  for  greater  attention  on  providing  safety  instruction  and 
improved  training  for  MRAP  drivers. 
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C.  MRAP  Training  Systems  Development 


1.  Perceived  Training  Need 

The  need  to  improve  training  for  these  new  types  of  vehicles  was  recognized  by  the 
Joint  MRAP  program  office  shortly  after  the  first  vehicles  were  fielded.  The  U.S.  Marine 
Corps  awarded  a  logistics  and  training  support  contract  in  November  2007  to  support 
“MRAP  University”  located  at  the  U.S.  Army’s  Red  River  Army  Depot,  Texarkana, 
Texas.  The  university  provided  a  schoolhouse  course  that  included  10  hours  of  classroom 
instruction  and  limited  “hands-on”  driving  instruction,  which  may  have  offered  little 
opportunity  to  practice  under  operational  driving  conditions.  Course  materials  were 
designed  for  both  operators  and  maintainers  (Weirauch  2011). 

The  U.S.  Marine  Corps  fielded  the  first  vehicle  trainer,  the  Common  Driver  Trainer. 
The  U.S.  Marine  Corps  driving  simulators  are  produced  by  FAAC  (Aerotech  Training 
and  Simulation  Division).  The  reconfigurable  simulators  are  placed  at  active  and  reserve 
U.S.  Marine  Corps  training  centers  and  bases  located  in  the  United  States. 

The  U.S.  Army  began  a  new  procurement  for  driving  simulators  in  2011  when  it 
awarded  a  $36  million  contract  to  Raydon  Corporation  to  build  1 1  MRAP  virtual  trainers. 
The  Anny  also  initiated  a  “MRAP  familiarization”  course  at  the  U.S.  Anny  McGregor 
Range.  This  course  is  a  1-day  event  that  covers  “general  maintenance  and  basic  operation 
of  the  vehicle.” 

2.  MRAP  Simulation  Training 
a.  Common  Driver  Trainer 

The  Common  Driver  Trainer  is  an  MRAP  vehicle  trainer  that  includes  high- 
resolution  visual  and  motion  simulations  representing  the  MRAP  vehicle  operational 
environment.  It  is  built  on  a  full-motion  base  that  rotates  in  all  axes  (six  degrees  of 
freedom).  The  system  displays  a  variety  of  virtual  world  scenes  that  simulate  on-  and  off¬ 
road  terrains  and  obstacles.  It  has  a  control  room  for  the  training  operator/instructor  and 
observer. 

The  Common  Driver  Trainer  can  conduct  scenario-based  training  under  varying 
environmental  conditions  (day-night,  sandstorm,  IED  contact/explosive  threats).  The 
training  includes  scenarios  designed  to  teach  avoiding  and  controlling  the  vehicle  in  a 
rollover  situation.  The  Common  Driver  Trainer  provides  driver  training  for  a  variety  of 
vehicles,  including  M1A2  Abrams,  Stryker,  and  MRAP  varieties,  using  reconfigurable 
vehicle  cab  and  software  routines.  The  Common  Driver  Trainer  was  fielded  in  November 
2007  and  is  now  used  by  the  U.S.  Marine  Corps  and  the  U.S.  Army. 
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b.  Operator  Driving  Simulator 

The  ODS  was  developed  to  accommodate  a  wide  variety  of  trucks — from 
HMMWVs  to  larger  heavy  tank  haulers.  The  simulator  uses  a  “generic”  vehicle  cab  that 
can  be  reconfigured  using  dashboard  kits  and  modular  software  changes. 

There  are  various  versions  of  the  simulator,  including  fixed-site  facility  or  self- 
contained  mobile  trailers.  Some  of  the  key  features  of  ODS  are  as  follows: 

•  ODS  can  be  fielded  in  several  configurations  (fixed  sites  and  trailer  options). 

•  Motion  cuing  systems  include  six-degree-of-freedom  full-motion  seats 

•  Visual  environments  are  scalable  and  range  from  180  to  225  degrees. 

•  FAAC  ODS  has  a  six-level  training  curriculum,  varying  in  degree  of  difficulty. 

•  FAAC  ODS  has  a  trainee-performance  tracking  system  that  monitors  and  scores 
trainee  progress  on  over  50  parameters. 

•  The  Instructor  Operator  Station  provides  display  information  regarding  trainee 
performance  status  and  provides  input  to  the  instructor  for  any  required  remedial 
training. 

An  expanded  version  of  ODS  was  developed  to  provide  two-crew  training  for 
missions  related  to  route  clearance.  This  version  includes  controls  and  display  kits  that 
enable  the  operator  to  operate  external  cameras  and  sensors.  Also  included  is  a  surface¬ 
scraping  simulation  of  terrain  defonnation  and  soil  disturbance. 

c.  Egress  Simulator 

The  MRAP  Egress  Trainer  (MRET)  is  designed  to  improve  the  ability  of  the  vehicle 
crew  to  quickly  and  safely  egress  from  a  rollover  event.  This  simulator  was  constructed 
in  response  to  lessons  learned  from  rollover  mishaps  indicating  that  inability  to  escape 
the  vehicle  after  rollover  was  a  key  factor  in  lethality  of  these  mishaps. 

The  MRET  simulates  rollovers  under  “dry  conditions”  and  trains  the  drivers  and 
crew  to  a  safe  egress,  considering  external  environment  and  vehicle  load.  Training  is 
accomplished  by  teaching  correct  crew  procedures  and  hazard  avoidance,  using  a 
combination  of  classroom  training  and  scenario-based  simulations.  Some  of  the  key 
features  of  the  MRET  include  the  following: 

•  MRAP  cab  is  interchangeable. 

•  Cab  is  on  a  frame  that  allows  360-degree  rotation. 

•  Crews  are  trained  to  egress  using  specific  crew-coordination  procedures. 

•  Scenario  training  includes  simulated  blocked  doors  and  damaged  vehicle  egress. 
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D.  MRAP  Training  System  Issues 


1.  MRAP  Acquisition  Issues 

Some  of  the  key  issues  regarding  MRAP  procurement  and  initial  deployment  are  as 
follows: 

•  Accelerated  deployment  schedule  was  required  to  meet  an  urgent  need  for 
protection  from  IED  threat,  overriding  the  systems  training  needs. 

•  Rapid  Acquisition  did  not  give  early  attention  to  incorporating  sustainment  in  the 
form  of  logistical  and  training  support  functions. 

•  MRAPs  designed  for  survivability  against  mines  and  IEDs  were  prone  to 
rollovers  and  also  were  difficult  to  egress  safely  following  a  rollover  mishap, 
necessitating  early  systems  training  and  simulators. 

•  Accident  rate  (mainly  rollovers)  was  too  high. 

•  Most  MRAP  drivers  did  not  attend  a  driving  school — most  had  to  learn  on  the  job 
(when  deployed) — and  standard  “truck  driver  training”  did  not  adequately  address 
key  operational  hazards  encountered  in  Iraq  and  Afghanistan. 

•  Drivers  had  little  or  no  opportunity  to  practice  driving. 

Following  an  analysis  of  numerous  rollover  mishaps,  the  U.S.  Marine  Corps  safety 
personnel  and  commanders  agreed  that  MRAP  drivers  should  follow  these  specific 
procedures  to  help  ensure  safe  operations  of  the  vehicles  (USMC  2008): 

•  Brief  route  familiarization,  including  road  and  terrain  hazards  before  the  mission. 

•  Conduct  a  pre-mission  inspection  of  vehicle  tire  pressure,  steering,  and  brakes. 

•  Know  your  vehicle  driving  performance  limitations. 

•  Be  especially  cautious  when  roadway  clearances  are  narrow,  traveling  at  or  near 
canals  and  rivers,  and  unimproved  roads. 

•  Avoid  hard  turns  and  sudden  maneuvers. 

•  Adjust  speed  to  conditions,  and  do  not  overcorrect  steering. 

•  Recognize  and  avoid  rollover  hazards  and  conditions. 

•  Maneuver  cautiously  to  avoid  rollover  or  collision  hazard. 

2.  Learning  Objectives 

Based  on  analysis  of  mishap  data  and  U.S.  Marine  Corps  guidelines  above,  here  is  a 
list  of  some  appropriate  top-level  training  objectives  that  could  have  been  enunciated  and 
used  as  a  point  of  departure  for  MRAP  training  system  development: 
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•  Given  description  of  on-road  or  off-road  conditions  the  student  will  be  able  to 
identify  hazards  and  rollover  risk.  (Knowledge  -  concept  -  classification) 

•  The  student  will  be  able  to  describe  procedures  for  driving  up  and  down  a  grade 
in  mountainous  terrain.  (Knowledge  procedure  -  task  steps) 

•  The  student  will  be  able  to  describe  procedures  for  driving  on  wet  and  muddy 
roadways,  off  road,  on  soft  road  shoulders,  near  riverbanks,  and  on  bridge 
crossings.  (Knowledge  procedure  -  task  steps) 

•  Given  various  photographic  or  video  examples  of  road  conditions,  the  student  will 
be  able  to  define  the  hazard  level  depicted.  (Knowledge  concept  -  definition  of 
concept  attributes) 

•  When  approaching  a  hazardous  road  or  off-road  condition,  the  student  will  be 
able  to  maneuver  the  vehicle  to  avoid  the  hazard  or  drive  to  lessen  the  risk  of 
rollover.  (Skill  -  pattern  recognition  -  complex  perceptual  motor  skill) 

•  The  student  will  be  able  to  avoid  or  recover  from  an  impending  rollover  event. 
(Skill  -  complex  perceptual  motor  skill) 

•  Following  a  rollover  event,  the  crew  will  be  able  to  coordinate  egress  task  activity 
and  follow  correct  procedures  for  safe  egress.  (Skill  -  individual  complex 
perceptual  motor  and  crew  communication) 

E.  MRAP  Summary  and  Case  Study  Implications 

This  chapter  summarizes  a  case  study  regarding  the  acquisition  of  MRAP  vehicles 
and  the  impact  of  delayed  training  system  development  for  the  MRAP  program. 

Most  of  the  casualties  in  Iraq  and  Afghanistan  were  due  to  injuries  sustained  from 
IEDS.  To  address  the  IED  threat,  the  DoD  initiated  a  program  of  rapid  acquisition  of 
MRAP  vehicles.  The  focus  was  on  vehicle  designs  that  protected  military  personnel 
against  the  IED  threat.  The  MRAP  rapid  acquisition  program  was  very  successful  in 
quickly  fielding  numerous  vehicles,  within  program  cost  and  schedule.  And  there  was 
little  question  that  the  MRAP  designs  provided  much  better  protection  against  IEDs.  But 
the  MRAP  vehicles  were  much  taller  and  heavier  than  those  previously  fielded  vehicles 
and  therefore  were  very  vulnerable  to  rollover  mishaps  that  resulted  in  extensive  material 
damage  and  numerous  fatalities.  The  mishap  rate  (mainly  rollovers)  was  too  high. 
MRAPs  had  three  times  the  rollover  mishap  rate  of  conventional  (unannored) 
HMMWVs.  The  MRAPs  designed  for  survivability  against  mines  and  IEDs  were  not 
only  prone  to  rollovers  but  also  were  difficult  to  egress  safely  following  a  rollover 
mishap. 

Safety  publications  and  GAO  and  congressional  reports  indicated  that  the  root  cause 
of  many  of  these  mishaps  was  traced  to  inadequate  training — soldiers  and  Marines  were 
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not  adequately  trained  to  drive  the  new  vehicles  and  were  not  prepared  to  encounter  the 
hazardous  driving  conditions  found  in  Iraq  and  Afghanistan. 

From  2007  to  2008,  which  represented  one  of  the  most  significant  vehicle  fielding 
time  periods,  there  were  122  MRAP  mishaps,  and  over  half  of  these  vehicle  mishaps 
were  rollovers.  The  vehicle  designs — relatively  higher  and  heavier  than  other  vehicles — 
contributed  to  the  mishap  rate.  For  example,  the  Caiman  MRAP  stands  9  feet  tall  and 
weighs  19  tons.  Other  factors  were  the  poorly  constructed  roads,  low  bearing  weight  river 
berms,  and  bridges  found  in  the  operational  theaters. 

Following  such  a  rash  of  mishaps,  military  leaders  expressed  concern  about  the 
adequacy  of  training  for  drivers.  The  U.S.  Marine  Corps  and  the  U.S.  Army  supported 
augmenting  training  with  additional  schoolhouse  training  to  address  the  issues  of  vehicle 
design  and  common  road  hazards.  Consideration  also  was  given  to  the  use  of  training 
simulators  that  could  model  selected  MRAP  vehicle  driving  characteristics  and  could 
train  to  scenarios  with  realistic  terrain  conditions. 

Training  development  started  after  the  vehicles  were  deployed,  and  it  followed 
operational  lessons  learned  by  addressing  both  augmented  driver  training  and  the  need  for 
rollover  training.  Two  major  simulator  developments  took  place:  the  ODS  and  the 
Common  Driver  Trainer.  Both  simulators  used  reconfigurable  truck  cabins  and  modular 
software,  and  both  simulators  trained  to  scenarios  that  included  hazardous  terrain 
conditions. 

It  did  appear,  however,  that  the  training  was  developed  more  or  less  ad  hoc  and 
without  significant  formal  DoD  sponsored  front-end  analysis  and  without  an  extensively 
documented  training  requirements  analysis.  And  while  the  simulators  are  equipped  with 
some  form  of  trainee  performance  monitoring,  it  is  not  clear  from  the  literature  whether 
or  not  the  training  is  “performance  or  competency”  based. 

The  MRAP  case  study  report  included  a  brief  overview  of  training  evaluation 
methods,  both  objective  evaluation  (training-transfer  experimental  methods)  and 
subjective  (training-attribute  instructional-quality  checklists).  The  evaluation  section  of 
the  report  was  included  as  a  point  of  departure  for  possible  follow-on  study  efforts  that 
would  cover  measurement  methods  and  metrics  for  evaluating  the  status  of  training  at 
different  phases  of  system  acquisition  and  for  assessing  the  quality  and  training 
effectiveness  following  implementation  and  operations.  Indeed,  it  is  suggested  that 
training  evaluation  continue  during  entire  life  cycle  of  the  weapon  system. 

The  overall  conclusion  of  the  MRAP  case  study  report  is  that  an  argument  can  be 
made  for  improving  weapon  system  acquisition  by  establishing,  funding,  and  enforcing  a 
disciplined  methodology  for  timely  and  systematic  training  development  and  timely 
delivery  of  training,  whether  or  not  that  acquisition  is  on  a  rapid  or  normal  time  scale. 
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It  is  therefore  recommended  that  follow-on  efforts  consider  addressing  the  issue  of 
identifying,  defining,  or  redefining  the  development  framework,  processes,  and  methods 
for  training  development  and  continual  training  system  evaluation. 

F.  Recommendations 

•  All  MDAPs  (including  rapid  acquisitions)  should  address  training  as  an  integral 
part  of  the  acquisition  program. 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

•  Any  MRAP  follow-on  efforts  should  include  a  comprehensive  training  systems 
plan  as  an  integral  part  of  the  program. 

•  Refer  to  Section  4.D.2.  for  a  list  of  system-specific  learning  objectives  for  future 
systems  development. 

•  Provide  training  with  the  fielded  system  as  a  package  to  realize  full 
potential/capability  of  new  systems. 
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5.  Husky  Mounted  Detection  System 
— Rebecca  Grier  and  Jennifer  Brooks 


A.  Husky  Mounted  Detection  System  Background 

1.  System  Description 

The  Husky  Mounted  Detection  System  (HMDS),  developed  for  the  Joint 
Improvised  Explosive  Device  Defeat  Organization,  is  being  used  by  route-clearance 
patrols  to  detect  mines  and  IEDs  in  support  of  Operation  Enduring  Freedom.  The  HMDS 
was  executed  as  a  rapid  acquisition  program  in  response  to  the  large  number  of  casualties 
that  occurred  in  Operation  Enduring  Freedom.  The  HMDS  was  first  deployed  in  2008.  A 
new  version  (First  Generation,  Second  Edition)  was  deployed  in  October  2011.  A  total  of 
180  systems  have  been  deployed  to  date,  with  a  third  edition  now  in  development.  Each 
combined  system  (HMDS  and  Husky)  costs  approximately  $500,000. 

B.  HMDS  System  Overview 

The  HMDS  (Figure  5-1)  consists  of  a  4-panel,  51-channel  ground-penetrating  radar 
(GPR)  mounted  at  the  front  of  a  Husky  vehicle  and  a  touch-screen  graphical  user 
interface  (GUI)  mounted  inside  the  cab  of  the  Husky.  During  route-clearance  missions  in 
theater,  the  single  occupant  (soldier  operator)  drives  the  Husky,  stops  upon  audio  alert  of 
a  suspected  target,  examines  the  GPR  data  using  the  GUI  mounted  in  the  cab  of  the 
Husky,  and  determines  whether  to  mark  the  suspected  target  for  further  interrogation.  If 
an  alert  is  further  investigated  it  slows  the  rate  of  advance  of  the  route-clearance  patrol.  If 
an  alert  is  ignored  the  Husky  or  another  vehicle  in  the  patrol  could  be  hit  by  a  mine  or 
IED.  The  Husky  and  the  HMDS  are  considered  separate  systems.  All  operators  of  HMDS 
must  be  trained  on  driving  the  Husky  before  receiving  training  on  HMDS.  This  case 
study  will  discuss  only  HMDS  training,  not  the  Husky  training. 
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Figure  5-1.  HMDS  in  field. 


C.  HMDS  Training  Systems  Development 

To  provide  deployed  military  personnel  a  desperately  needed  capability  as  quickly 
as  possible,  the  HMDS  was  developed  using  rapid  acquisition.  Unlike  in  traditional 
acquisition,  in  rapid  acquisition  the  role  of  the  life-cycle  logistician  is  not  well  defined. 
One  consequence  of  this  for  HMDS  was  that  no  training  plan  was  developed  before  the 
first  systems  were  deployed  in  2008,  although  training  has  evolved  since  then.  The  time 
line  of  this  evolution  is  represented  in  Figure  5-2. 

The  HMDS  training  initially  only  occurred  in  theater  and  was  conducted  by  the 
original  equipment  manufacturer.  Although  the  training  was  scheduled  to  last  1  week, 
soldiers  were  often  pulled  away  from  HMDS  training  to  complete  other  operational  tasks 
or  training.  In  some  cases,  the  soldiers  were  receiving  no  training  on  HMDS  before 
operating  it.  If  soldiers  received  the  entire  week  of  training,  they  began  with  a  day  of 
classroom  training,  followed  by  4  days  in  the  actual  system.  Over  the  course  of  training 
(both  classroom  and  in  vehicle),  soldiers  spent  approximately  4  hours  on  troubleshooting, 
and  24  hours  on  threat  detection  and  identification;  the  remaining  time  was  spent  on 
system  constraints,  GUI  operation,  and  vehicle  operations.  Although  the  majority  of 
soldiers  are  certified  Husky  operators  before  training,  the  GPR  panels,  which  are 
mounted  to  the  Husky  for  IED  detection  operations,  significantly  change  the  performance 
of  the  vehicle.  As  a  result,  some  training  is  devoted  to  learning  to  drive  the  Husky  with 
the  GPR  panels. 
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Figure  5-2.  Training  development  time  line. 


The  student-teacher  ratio  for  this  training  was  4:2,  but  instruction  while  operating 
the  HMDS  in  an  actual  Husky  is  one  on  one.  Since  the  Husky  cab  only  seats  one  person, 
for  instruction  to  occur,  the  soldier  sits  in  the  cab  of  the  Husky  with  the  overhead  doors 
open,  and  the  instructor  is  outside  the  cab  leaning  into  the  vehicle.  Instruction  covers  the 
following:  (1)  driving  the  Husky  with  the  mounted  GPR  panels,  (2)  detection  and 
identification  of  threats,  (3)  maintenance  troubleshooting,  (4)  use  of  GUI,  and  (5)  how 
system  constraints  affect  operation  in  different  environments  (e.g.,  speed,  terrain,  etc.). 
When  the  system  was  originally  deployed,  the  soldiers  who  were  not  in  the  Husky  waited 
for  their  turns.  On  average,  soldiers  would  receive  8  hours  of  instruction  in  the  Husky 
during  the  4  days  of  in-vehicle  instruction. 

The  training  enhancements  that  have  occurred  since  2008  have  made  training 
available  before  deployment  and  improved  the  quality  and  cost  effectiveness  of  the 
training.  The  majority  of  operators  are  now  receiving  training  before  deployment.  The 
first  major  enhancement  to  the  training  was  the  development  of  familiarization  training 
for  the  National  Guard  and  Army  reserves.  This  training  was  first  delivered  at  Ft.  McCoy 
in  2009  (a  year  after  the  HMDS  was  initially  deployed).  This  training  did  not  replace  the 
in-theater  training,  but  the  familiarization  training  increased  the  likelihood  that  operators 
had  some  training  before  operating  the  system.  In  some  cases,  one  soldier  received  the 
training  and  then  another  soldier  was  selected  to  be  the  in-theater  operator. 

The  next  development  in  the  training  of  HMDS  did  not  occur  until  January  2011, 
with  the  fielding  of  surrogate  systems.  Before  this  development,  millions  of  dollars  were 
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spent  replacing  panels  damaged  during  training,  when  the  operators  would  run  the  panel 
into  the  ground  because  they  were  unfamiliar  with  how  the  Husky  handled  with  panels 
for  the  HMDS  mounted.  Each  of  the  four  GPR  panels  in  the  HMDS  costs  $40,000.  The 
surrogate  systems  fielded  in  January  2011  cost  only  $15,000  each,  and  an  individual 
surrogate  system  panel  costs  $1000.  Currently,  26  surrogate  systems  are  fielded,  but  the 
plan  is  to  field  a  total  of  55  systems. 

The  detection  capability  of  the  surrogate  panels  is  done  with  software,  since  there  is 
not  any  actual  detection  capability  in  them.  Also  in  the  surrogate  system,  there  is  software 
that  enables  an  instructor  to  introduce  up  to  10  various  targets  to  the  operator. 
Specifically,  when  the  operator  gets  to  a  predetermined  point,  the  system  provides  an 
alarm  and  presents  an  image  similar  to  what  the  operator  would  see  on  an  actual  system. 
The  operator  then  makes  a  detennination  of  “target”  or  “no  target.”  The  instructor  has 
three  methods  available  for  selecting  targets:  radio-frequency  identification  tag,  GPS- 
based,  and  “tick  wheel”  or  distance-based.  All  three  methods  are  available  for  use  in  each 
vehicle. 

The  third  development  in  HMDS  training,  interactive  multimedia  instruction,  was 
fielded  in  February  2011.  Interactive  multimedia  instruction  is  delivered  via  CD  and 
enables  the  operator  to  review  system  start-up  procedures,  preventive  maintenance 
checks  and  services,  as  well  as  GPR  scans  of  threats  that  have  been  encountered  by  other 
operators.  This  training  can  be  completed  anytime,  anywhere  and  provides  soldiers  who 
are  waiting  for  their  turn  in  the  vehicle  a  chance  to  interact  with  HMDS.  It  also  increases 
exposure  to  a  wider  diversity  of  GPR  scans  (false  alarms  and  threats)  during  training. 

In  April  2011,  the  next  development  in  training,  the  virtual  route  clearance  trainer, 
was  fielded.  The  virtual  trainer  exists  in  four  tractor  trailers,  each  with  multiple  stations 
inside.  Some  or  all  of  the  26  positions  in  a  route-clearance  patrol,  including  the  HMDS 
operator,  can  participate  in  simulation-based  training  that  mirrors  real-world  variables 
such  as  terrain,  weather,  IEDs  (including  smoke,  fire,  and  explosions),  and  friend  and  foe 
vehicles.  Three  more  virtual  route-clearance  trainers  are  in  development.  In  addition,  the 
latest  version  of  Virtual  Battle  Space  (VBS2),  a  gaming  system  used  by  the  military  to 
expand  vehicle  training,  included  HMDS  as  one  of  the  vehicle  systems  the  gamer  can 
select.  The  version  of  VBS2  with  HMDS  was  deployed  in  July  2011. 

Finally,  in  fall  2011,  a  full  operator-certification  course  and  the  first  maintainer- 
certification  course  were  offered  at  Counter  Explosives  Hazard  Center,  Ft.  Leonard 
Wood,  MO.  Operator  training  is  still  provided  in  theater  as  described  above.  The  operator 
certification  course  now  offered  at  Ft.  Leonard  Wood  is  similar  to  the  training  described 
above  as  well,  but  the  student-teacher  ratio  is  8:3. 
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D.  HMDS  Training  System  Issues 

The  training  that  was  provided  upon  initial  deployment  was  not  a  mature  training 
concept.  It  took  1  year  from  initial  deployment  for  any  improvements  in  the  training.  It 
took  a  further  2  years  for  an  operator  certification  course  to  be  offered  in  the  United 
States.  This  delay  in  training  resulted  in  an  HMDS  that  was  not  as  capable  as  it  could  be 
in  the  hands  of  a  well-trained  crew,  and  the  life-cycle  costs  increased  due  to  damage  to 
expensive  components  in  early  operational  training. 

1.  Impact  on  Performance 

Initially,  the  only  training  occurred  when  soldiers  arrived  in  theater.  This  training 
was  limited  by  the  constraints  of  the  vehicle,  as  well  as  the  operational  environment. 
Training  was  further  interrupted  by  other  demands  pulling  soldiers  away  from  training. 
Therefore,  soldiers  were  often  operating  the  system  with  minimal  training.  Some  soldiers 
who  did  not  receive  a  complete  training  cycle  believed  that  the  primary  benefit  of  the 
system  was  to  take  a  hit  by  a  mine  or  IED  and  not  to  detect  threats  (stated  by  one  soldier 
during  an  operational  test  of  HMDS). 

Even  if  the  soldiers  did  believe  that  the  primary  goal  of  HMDS  was  to  detect  threats, 
due  to  lack  of  training  areas  or  simulators,  they  had  limited  exposure  to  the  variety  of 
threats  and  false  alarms  that  could  be  encountered.  As  a  result,  their  ability  to  determine 
the  nature  of  a  threat  was  impaired.  Interactive  multimedia  instruction  increases  soldiers’ 
exposure  to  different  GPR  scans,  which  improves  their  ability  to  distinguish  between 
threats  and  false  alarms. 

2.  Life-Cycle  Costs 

The  lack  of  mature  training  not  only  affected  soldier  performance,  it  also  affected 
the  life-cycle  cost  of  HMDS.  The  student-teacher  ratio  was  reduced  from  4:2  to  8:3, 
which  lowered  the  cost  of  the  training.  In  addition,  there  are  now  several  options 
(including  the  VBS2,  the  virtual  trainer,  and  the  interactive  multimedia  instruction)  for 
increasing  operator  exposure  to  HMDS  features. 

As  noted  above,  soldiers  who  did  not  receive  proper  instruction  on  the  system 
operated  the  system  as  if  it  were  to  take  the  brunt  of  a  mine  or  IED.  Each  Husky-HMDS 
costs  $500,000.  Training  soldiers  to  understand  the  most  effective  way  of  using  the 
system  saves  the  cost  of  replacement  systems. 

The  GPR  panels  are  an  expensive  and  delicate  subsystem  of  HMDS.  When  soldiers 
were  learning  to  drive  the  Husky  with  the  installed  panels,  many  panels  were  damaged 
(e.g.,  some  soldiers  ran  the  panels  into  the  ground  while  driving  through  a  ditch). 
According  to  NIITEK,  the  manufacturer,  millions  of  dollars  were  being  spent  every  few 
months  to  replace  damaged  panels.  Surrogate  systems  have  been  fielded  that  can  be  used 
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to  instruct  on  driving  with  the  panels.  Besides  costing  less  than  the  GPR  panels,  these 
surrogate  panels  are  much  studier.  As  a  result,  they  are  replaced  with  less  frequency  than 
the  fully  functioning  panels. 

E.  HMDS  Summary  and  Case  Study  Implications 

It  took  2  years  for  proper  HMDS  training  to  be  developed.  Because  training 
development  did  not  begin  until  1  year  after  deployment,  it  took  3  years  for  HMDS  to 
reach  its  full  effectiveness.  Not  only  is  this  a  disservice  to  our  military  personnel,  it  is  an 
inefficient  use  of  program  funds.  All  acquisition  programs  must  recognize  that  full 
system  effectiveness  will  not  be  reached  until  a  training  plan  is  in  place  to  achieve  the  full 
system  operational  capabilities. 

F.  Recommendations 

•  All  MDAPs  (including  rapid  acquisitions)  should  address  training  as  an  integral 
part  of  the  acquisition  program. 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

•  “Right  size”  student-to-trainer  ratio. 

•  Allow  for  training  to  be  completed  without  interruptions. 

•  Provide  training  with  the  fielded  system  as  a  package  to  realize  full 
potential/capability  of  new  systems. 
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6.  P-8A  Poseidon 

— Charles  G.  Sanders 


A.  Poseidon  Background 

The  P-8A  Poseidon  (Figure  6-1)  is  a  long-range  anti-submarine  warfare;  anti¬ 
surface  warfare;  intelligence,  surveillance,  and  reconnaissance  aircraft  capable  of  broad- 
area,  maritime  and  littoral  operations.  The  P-8  A  will  be  a  multi-mission  aircraft,  unlike 
the  single-mission  design  of  the  P-3C.  This  broader  mission  responsibility  adds  to  the 
complexity  of  crew  training  and  is  discussed  in  more  detail  later  in  this  chapter.  The  U.S. 
Navy  plans  to  purchase  117  P-8As  (a  modified  Boeing  737)  to  replace  its  fleet  of  P-3C 
aircraft.  The  first  test  aircraft  began  formal  flight  testing  in  late  2009.  Initial  operational 
capability  is  slated  for  2013. 


Source:  http://www.boeinq.com/defense-space/military/p8a/docs/P-8A  overview.pdf. 

Figure  6-1.  P-8A  Poseidon. 

The  P-8A  will  conduct  rotational  deployments,  consistent  with  the  Fleet  Response 
Plan,  consisting  of  three  6-month  phases  (Basic,  Intermediate,  and  Deployed).  In 
November  2005,  the  Navy  announced  that  the  P-8  A  preliminary  design  review  (PDR), 
conducted  from  October  31  through  November  4,  was  the  best  major  weapons  system 
PDR  it  had  ever  reviewed;  however,  there  was  no  mention  of  training  or  simulators  in  the 
document.  A  successful  critical  design  review  was  completed  in  July  2007.  Dates  for  the 
Operational  Test  and  Evaluation,  a  significant  stage  in  the  acquisition  process  where 
system  training  is  evaluated,  were  TBD  as  of  May  2012. 
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B.  Poseidon  System  Overview 

To  determine  the  current  developmental  status  of  the  P-8A,  a  number  of  reports 
were  reviewed  and  are  summarized  below. 

1.  Technology  Development  Strategy  (January  2012) 

The  P-8A  Poseidon  program  employs  an  evolutionary  acquisition  approach,  with 
Increment  1  planned  to  achieve  Milestone  C,  with  full  rate  production  and  initial 
operating  capability  planned  for  FY13.  Increment  2  is  currently  being  managed  as  an 
Engineering  Change  Proposal,  with  planned  introduction  in  FY16.  This  Technology 
Development  Strategy  applies  to  the  third  increment.  As  a  follow-on  Increment  of  the  P- 
8A  program,  this  effort  is  designed  to  be  a  modification  to  the  weapon  system  that  is 
expected  to  be  operationally  fielded  by  FY13. 

The  Increment  3  modification  will  provide  both  new  and  improved  capability  to  the 
warfighter.  The  P-8A  Increment  3  program  intends  to  maximize  the  use  of  mature 
technologies,  including  non-developmental  items,  government-off-the-shelf  technologies, 
and  commercial-off-the-shelf  technologies  to  reduce  cost,  schedule,  and  program  risk. 

The  program  will  build  on  an  open-application  framework  to  provide  an  integration 
layer  between  the  baseline  aircraft  architecture  and  capability  subsystems.  This  approach 
is  designed  to  take  advantage  of  open-source  development  standards  to  produce 
application  consistency  with  increased  portability  while  maximizing  competitive 
acquisition  approaches  and  best-of-breed  solutions. 

P-8A  Increment  3  is  not  a  new  start  program;  it  is  budgeted  as  a  Project  Unit  (3218) 
within  the  P-8A  Poseidon  Project  Element  (0605500N).  The  P-8A  baseline  and  defined 
Increments  are  described  as  Navy-only  programs  with  joint  potential  in  the  future. 

The  evolutionary  acquisition  strategy  was  established  and  approved  at  Milestone  B 
for  the  P-8A  program  in  2004.  The  delivery  schedule  for  Increment  3  is  based  on  the 
delivery  of  the  baseline  program  in  FY13  and  Increment  2  delivery  in  FY16. 

Following  the  Milestone  A  decision,  the  program  entered  the  Technology 
Development  phase,  followed  by  Engineering  and  Manufacturing  Development  entry  at 
Milestone  B.  The  strategy  for  production  and  deployment  of  Increment  3  is  based  on 
retrofit  of  the  fielded  P-8  A  weapon  system.  This  is  expected  to  begin  in  FY19,  following 
the  completion  of  Integrated  Test. 

The  details  of  program-sustainment  strategy  are  included  in  the  Life  Cycle 
Sustainment  Plan,  which  will  be  updated  to  include  Increment  3  capabilities  as  required. 
This  indicates  that  plans  for  training  of  P-8A  crew  are  not  finalized.  Plans  for  the  P-8A 
major  contracts  described  in  this  document  do  not  include  the  training-development 
contract. 
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2.  Defense  Acquisition  Executive  Summary  (February  2012) 

The  Program  Manager’s  estimate  for  full  rate  production  has  changed  from  April 
2013  to  June  2013.  This  change  is  the  result  of  a  2-month  delayed  start  of  Live  Fire  Test 
and  Evaluation  due  to  test  asset  availability.  The  program  is  currently  fully  funded. 

The  Defense  Acquisition  Executive  Summary  reported  delays  in  integrated  test 
execution,  and  the  T&E  rating  remains  Yellow  against  the  contracted  plan.  Due  to 
inefficiencies  in  test  execution,  the  program  has  not  yet  achieved  execution  of  the 
integrated  test  program  per  the  plan.  The  Initial  Operational  Test  and  Evaluation  start 
date  was  moved  from  April  2012  to  June  2012  based  on  demonstrated  test  execution 
performance.  Although  early  P-8A  flight-test  instrumentation  and  flight-test  maintenance 
immaturity  slowed  initial  progress,  flight-test  efficiencies  have  improved. 

3.  Program  Assessment  Report  (February  2012) 

The  Training  Systems  and  Government  Services  Division  of  Boeing  continues  the 
process  of  planning  to  optimize  the  overall  schedule  by  making  changes  affecting  a 
combination  of  Contract  Line  Item  Number  0503  and  LRIP-Phase  2.  The  Training 
Systems  and  Government  Services  Division  is  also  evaluating  workaround  plans  to 
improve  the  Part  Task  Trainer-Phase  delivery.  There  were  no  known  training  issues  noted 
in  this  report. 

C.  Poseidon  Training  Systems  Development 

The  Navy  Training  Systems  Plan  (NTSP)  dated  June  2010  was  reviewed  by  CNO 
staff  for  approval.  Based  on  the  NTSP  dated  June  2010,  the  P-8  A  training  program  will 
consist  of  initial  and  follow-on  training  for  operator  and  maintenance  personnel.  Enlisted 
Air  crewman  Operator  training  will  be  provided  by  Center  for  Naval  Aviation  Technical 
Training  Detachment  (CNATT  DET)  Kaneohe  Bay,  which  currently  provides  this 
training  for  the  P-3A.  CNATT  Unit  (CNATTU)  Jacksonville  will  provide  enlisted 
maintenance  training  for  the  P-8A.  P-3A  maintenance  training  is  currently  provided  by 
CNATTU  Whidbey  Island  and  CNATTU  Jacksonville. 

Navy  will  utilize  a  combination  of  full-motion  simulators,  weapons  tactical  trainers, 
and  aircrew  part  task  trainers  for  P-8A  crew  training.  There  will  also  be  a  deployable 
mission  readiness  trainer  and  maintenance  trainers.  This  mix  is  consistent  with  other 
programs  as  an  appropriate  means  to  balance  capability  and  cost. 

The  NTSP  states  that  the  goal  is  to  satisfy  90%  of  all  P-8A  aircrew  training  and 
Training  and  Readiness  requirements  in  simulators  or  in  a  classroom  environment.  This  is 
a  significant  increase  over  past  programs,  which  historically  rely  on  live  training  venues 
for  meeting  the  Training  and  Readiness  matrix  requirements.  Initially,  Boeing  will 
provide  Interim  Contractor  Support  for  the  aircraft  training  system. 
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The  instructor  cadre  will  consist  of  contractors  and  naval  personnel,  with  Navy 
personnel  providing  training  on  tactics,  operational  employment,  and  check 
ride/certification  events. 

Initial  training,  primarily  in  support  of  developmental  testing  and  evaluation,  is  in 
progress. 

The  plan  is  to  transition  squadrons  from  P-3C  to  P-8A  during  inter-deployment 
readiness  cycle.  Follow-on  training  for  pilots  and  Naval  Flight  Officers  will  be  provided 
by  VP-30,  the  training  squadron  for  the  P-3  community.  Air  crewman  Operator  training 
will  be  provided  by  CNATT  DET  Kaneohe  Bay.  Maintenance  training  will  be  provided 
by  CNATTU  Jacksonville. 

D.  Poseidon  Training  Systems  Issues 

Five  findings  discovered  during  analysis  of  the  P8A  as  a  case  study  are  discussed 
below  with  associated  concerns  and  any  follow-up  actions. 

1.  Training  Simulators  are  Not  Networked 

a.  Finding 

The  NTSP  indicates  that  all  major  training  elements  of  DoD  Directive  1430.13, 
Training  Simulators  and  Devices,  will  be  addressed.  It  also  states  that  training  plans  will 
maximize  new  learning  technologies,  simulation  technology,  embedded  training,  and 
instrumentation  systems  to  provide  anywhere,  anytime  training. 

b.  Concern 

There  is  no  mention  of  networking  P-8A  training  systems  with  other  platforms  to 
facilitate  battle  group  or  joint  training.  This  should  be  at  least  an  implicit  requirement, 
since  there  is  also  no  mention  of  battle  group,  fleet,  or  joint  task  force  training  that 
addresses  how  the  P-8A  will  be  incorporated  into  the  fleet  or  the  impact  that  the  P-8A 
will  have  on  fleet  training.  Without  the  ability  to  network  P-8A  training  simulators  to 
other  platform  simulators,  the  only  way  to  train  as  a  battle  group  is  during  live  training  on 
the  training  range.  Informally,  we  have  been  informed  by  PMA-205  that  they  will  be 
adding  the  capability  to  link  the  simulators. 

c.  Follow-up 

In  a  phone  conversation  with  PMA-205  (on  26  March  2012),  it  was  found  that  the 
Navy  has  plans  for  the  P-8A  simulators  to  link  into  the  Fleet  Synthetic  Training  Joint 
events  through  the  NCTE.  It  was  further  stated  that  an  NCTE  node  is  installed  in 
Jacksonville,  with  future  plans  to  install  nodes  in  the  other  simulator  locations.  When 
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asked  why  these  plans  are  not  addressed  in  the  NTSP,  it  was  stated  that  the  latest  version 
(dated  June  2010)  is  under  revision,  with  a  draft  release  for  review  expected  in  the 
summer  of  2012. 

2.  Training  Packages  Lag  System  Development 

a.  Finding 

The  NTSP  highlighted  one  negative  aspect  of  the  training  concept,  in  that  if  four 
squadrons  are  transitioned  it  would  require  hands-on  labs  to  be  conducted  on  LRIP 
aircraft.  This  issue  is  due  to  the  revision  to  the  transition  plan,  which  is  better 
synchronized  with  construction  of  a  new  building,  but  delivers  training  packages  about  a 
year  later  than  originally  planned. 

b.  Concern 

Because  of  the  requested  SDD  Training  System  Alternative  Analysis,  decisions  on 
use  of  mission  Part  Task  Trainers,  Integrated  Avionics  Trainers,  and  Deployable  Mission 
Readiness  Trainers  is  not  finalized.  This  adds  risk  to  the  effectiveness  of  the  NTSP. 

3.  Contractor  Determines  Maintenance  Training  Requirements 

a.  Finding 

Due  to  the  change  in  maintenance  concept  from  Contractor  Logistics  Support  to 
organic,  the  number  and  type  of  organic  maintenance  courses  have  yet  to  be  determined. 
A  Training  Situation  Analysis  is  being  conducted  by  Boeing  to  detennine  required 
maintenance  training  for  the  P-8A. 

b.  Concern 

It  is  unclear  why  the  Navy  would  ask  the  contractor  to  analyze  the  training 
requirements,  instead  of  having  the  Navy  training  community  do  the  analysis.  It  is  the 
understanding  of  the  study  group  that  the  Navy  training  community  should  be  better 
qualified  to  determine  the  training  requirements  for  the  P-8A. 

4.  New  Missions  Imply  New  Training  Requirements 
a.  Finding 

With  the  increase  in  mission  areas  for  the  P-8A  over  the  P-3C,  the  Training  and 
Readiness  matrix  will  be  enlarged. 
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b.  Concern 

The  NTSP  does  not  address  whether  or  not  the  crew  will  have  any  challenges  in 
satisfying  the  increased  training  requirements  or  how  the  increase  will  be  handled. 

c.  Follow-up 

When  PMA-205  was  asked,  current  intent  was  stated  (phone  conversation,  26 
March  2012)  that  the  focus  is  primarily  on  three  mission  areas,  with  first  priority  placed 
on  the  anti-submarine -warfare  mission  and  secondary  time/resources  dedicated  to  anti¬ 
ship  and  intelligence,  surveillance,  and  reconnaissance  missions.  The  experience  of  the 
study  team  is  that  primary  focus  on  anti-submarine  warfare  is  not  consistent  with  the  fleet 
training  trend  or  Combatant  Command  requirements  over  the  past  several  years. 

5.  Manpower  Requirements  for  Logistics  Support  Increased 

a.  Finding 

At  contract  award,  the  Program  of  Record  decision  for  P-8A  organizational  level 
maintenance  was  established  as  full  Contractor  Logistics  Support.  However,  in  April 
2007,  Boeing  submitted  Contractor  Logistics  Support  manpower  requirements,  via 
Contract  Data  Requirements  List  A00W-147,  which  showed  a  substantial  increase  in  the 
manpower  requirements  over  its  November  2004  Contract  Data  Requirements  List 
delivery  estimate. 

b.  Concern 

This  may  cause  a  significant  impact  on  training  resources.  These  increased 
requirements  resulted  in  a  corresponding  increase  in  the  total  life-cycle  cost  estimate  for 
the  P-8A  Program.  This  prompted  the  Program  Manager  to  initiate  a  formal  Business 
Case  Analysis  to  detennine  the  most  cost-effective,  best  value  approach,  with  an 
acceptable  risk  for  P-8A  organizational-level  maintenance  manpower.  A  Manpower 
Working  Group  was  established  to  perform  the  Business  Case  Analysis. 

New  Additional  Qualification  Designation  codes  were  developed  for  Navy  pilots 
and  Naval  Flight  Officers  assigned  to  P-8A  squadrons.  In  addition,  new  planned  Navy 
Enlisted  Classification  codes  were  developed  for  P-8A  maintenance  personnel.  The 
manpower  requirements  are  significantly  less  than  what  is  programmed  in  the  FDYP  for 
the  P-3C.  The  P-8A  will  have  a  9-member  crew,  in  contrast  to  the  11 -member  crew  for 
the  P-3C.  Note  that  this  crew  reduction  comes  in  spite  of  the  broader,  multi-mission  roles 
of  the  P-8A,  which  will  require  the  logically  broader  training  associated  with  the 
expanded  mission  capabilities. 
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E.  Poseidon  Summary  and  Case  Study  Implications 

The  P-8A  training  program  will  consist  of  initial  and  follow-on  training  for 
operators  and  maintainers.  Due  to  the  change  in  maintenance  concept  from  Contractor 
Logistics  Support  to  all  organic,  the  number  and  type  of  organic  maintenance  courses  had 
not  been  determined  at  the  time  of  publishing  this  NTSP.  Boeing  completed  the  Training 
Situation  Analysis  in  February  2010  and  is  currently  (spring  of  2012)  in  the  process  of 
conducting  a  Fidelity  Analysis  to  further  define  the  best  courses  of  action  for 
maintenance  training. 

A  Training  Situation  Analysis  is  one  of  four  major  analysis  activities  of  the  Navy 
Training  Systems  Requirements  Analysis,  which  is  how  the  Navy  satisfies  Department  of 
Defense  Directive  1430.13,  Training  Simulators  and  Devices.  It  is  intended  to  be  the 
front-end  analysis  for  training  requirements.  The  other  three  analysis  activities  are 
Training  System  Alternatives  Report,  Training  Device  Requirements  Document,  and 
Training  System  Functional  Description. 

F.  Recommendations 

•  Add  the  requirement  to  provide  network  connectivity  and  logical  interoperability 
for  the  suite  of  P-8A  simulators  to  ensure  battle  group,  fleet,  and  joint  task  force 
training  scenarios  are  fully  supported  and  integrated  with  the  live,  virtual,  and 
constructive  training  environments. 

•  Require  analysis  of  the  training  load  associated  with  the  increase  in  mission  areas 
to  verily  if  the  crew  will  actually  have  time  and  resources  to  satisfy  all  the 
mission  and  training  requirements. 

•  Synchronize  scheduling  of  simulators,  training  tools,  and  packages  to  be  available 
to  train  for  first  deployment  to  ensure  full  system  capabilities  are  available  for 
operations. 

•  Expedite  any  changes  to  logistics  and  maintenance  plans  to  include  the  full 
complement  of  maintenance  personnel  and  training  to  define  the  best  courses  of 
action. 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 
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7.  Findings  and  Recommendations 


A.  Findings 

The  training  systems  acquisition  study  has  met  the  research  objective  of  providing 
evidence  from  a  range  of  case  study  programs  to  reinforce  the  need  for  training  systems 
considerations  to  be  incorporated  early  in  the  major  Defense  programs  and  on  a 
continuing  basis  through  the  life  cycle  of  a  given  system.  The  objective  of  this  task  is  to 
identify  and  analyze  the  specific  benefits  of  early  and  effective  incorporation  of  training 
details  into  acquisition  programs,  particularly  those  with  significant  human  systems 
interface  requirements.  The  research  provides  an  empirical  base  for  the  DoD  training  and 
acquisition  communities  to  understand  and  justify  the  need  to  plan  for  efficient  and 
effective  operator,  maintainer,  and  leadership  training  early  and  continually  in  the 
acquisition  process  to  improve  the  resulting  capabilities  and  utility  of  future  complex 
weapon  systems.  The  study  results  complement  the  latest  DoD  acquisition  and  training 
policies. 

The  Patriot  Program,  with  more  than  20  years  of  history,  provides  a  strong  case  for 
upgrading  training  in  the  complex  programs  on  a  continuing  basis  through  the  system  life 
cycle.  The  case  study  includes  evidence  from  multiple  sources  that  as  the  significant 
technical  upgrades  were  perfonned  over  the  years,  the  training  tools  and  learning  content 
were  not  enhanced  to  optimize  the  effectiveness  of  the  evolutionary  improvements  to  the 
system.  As  a  general  rule,  systems  upgrades  should  be  accompanied  with  corresponding 
changes  to  the  training  content  at  all  levels. 

The  Army  FCS  program  provides  a  useful  case  study  from  a  training  perspective. 
Two  aspects  of  the  program  stand  out  positively.  The  FCS  program  included  training  as  a 
KPP,  and  as  a  family  of  net-centric  weapons  systems  it  provided  netted  infrastructure  to 
host  embedded  training  across  the  many  programs  and  aspects  of  the  system  of  systems. 
The  Army  recognized  embedded  training  as  an  integral  and  essential  capability  for  the 
entire  family  of  systems.  The  FCS  is  considered  to  be  a  good  model  of  training 
development  for  future  MDAPs  for  three  reasons:  (1)  training  was  stressed  from  the 
outset — training  was  included  as  a  KPP;  (2)  embedded  training  was  integrated  into  the 
FCS  operational  hardware  and  software;  and  (3)  material  developers  worked  with 
training  developers  to  monitor  the  maturity  of  technologies  and  incorporate  those 
technologies  into  the  implementation  of  training. 

The  MRAP  program  was  a  rapid  acquisition  and  therefore  not  subject  to  the 
deliberate  acquisition  planning  process  evidenced  in  our  other  case  studies.  However, 
with  the  rapid  production  and  deployment  of  the  vehicles  in  2007-2008,  the  operators 
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began  to  experience  vehicle  rollovers  early  on — some  60  MRAP -reported  mishaps 
between  November  2007  and  June  2008,  with  over  one -half  of  those  attributed  to 
rollovers.  Lack  of  training  was  identified  in  the  narratives  for  the  most  serious  incidents 
60%  of  the  time.  The  need  to  improve  training  for  these  new  types  of  vehicles  was 
recognized  early  by  the  Joint  MRAP  program  office.  A  series  of  driver 
trainers/simulators  was  developed  as  expeditiously  as  possible,  but  the  lesson  learned 
once  again  is  to  develop  training  in  concert  with  systems  development — even  in  rapid 
acquisitions. 

The  HMDS  is  another  rapid-development  program  with  some  of  the  same  training 
lessons  learned  as  those  found  in  the  MRAP  program.  It  took  2  years  for  robust  HMDS 
training  to  be  developed,  with  training  development  beginning  a  full  year  after 
deployment,  and  it  took  3  years  for  the  HMDS  training  to  reach  full  effectiveness. 

The  Navy  P-8A  Poseidon  program  provides  a  useful  case  study  from  several 
different  perspectives.  First,  it  is  a  multi-mission  aircraft  replacing  the  single-mission 
design  of  the  P-3C.  The  Poseidon  is  a  modified  Boeing  737,  which  has  had  a  mature  life 
cycle  in  commercial  use  dating  back  to  the  1960s.  Among  the  findings  in  this  case  study 
was  the  indication  that  the  P-8A  training  simulators  are  not  networked  to  other  platforms 
to  facilitate  training  for  battle  group,  fleet,  or  joint  task  force  operations.  The  change  in 
maintenance  concept  by  the  manufacturer  resulted  in  an  increased  load  on  Navy  training, 
with  analysis  underway  to  define  the  best  courses  of  action  for  maintenance  training. 

A  number  of  acquisition  and  readiness  issues  are  related  to  a  full  scope  of  systems 
training  across  the  lifetime  of  any  given  program.  Evidence  has  been  documented  here  to 
expect  more  effective  and  efficient  operations  and  lower  system  life-cycle  costs  through 
consideration  of  training  system  acquisitions  as  an  integral  part  of  every  MDAP 
development. 

B.  Recommendations 

The  case  studies  in  this  report  reinforce  the  value  of  determining  comprehensive 
training  needs  over  a  program  life  cycle.  Training  simulators  and  learning  content  for  the 
system  operators  and  maintenance  personnel  should  be  developed  before  initial 
introduction  of  the  hardware  system.  Detailed  recommendations  from  this  research  are 
listed  below: 

•  Move  training  assessment  to  the  left  (earlier)  in  the  acquisition  cycle. 

-  Revise  Defense  and  service  acquisition  policies  to  require  earlier 
assessments  of  training. 

•  Training  assessments  should  be  early  and  continual. 
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System  upgrades  require  training  upgrades;  improvements  in  systems  hardware 
and  software  must  be  accompanied  by  corresponding  training  enhancements  at  all 
organizational  levels. 

All  MDAPs  (including  rapid  acquisitions)  should  address  training  as  an  integral 
part  of  the  acquisition  program. 

Impacts  to  training  concepts  and  systems  and  devices  should  be  part  of  the 
weapon  systems  development  decisions. 

Integrate  embedded  training  with  MDAP  weapons  systems  as  feasible.  Net¬ 
centric  systems  of  the  future  provide  special  opportunities  to  build  in  embedded 
training. 

Monitor  technical  maturity  to  incorporate  parallel  critical  training  technologies. 

Provide  training  with  the  fielded  system  as  a  package  to  realize  the  full  capability 
of  new  systems. 

-  Training  plans  should  be  developed  well  before  successive  program 
milestones  leading  to  the  initial  fielding  of  the  hardware  system,  and 
similarly  updated  and  packaged  with  every  fielded  system  modification. 

Require  MDAPs  to  provide  network  connectivity  and  logical  interoperability  for 
the  suite  of  simulators  and  training  tools  so  training  scenarios  are  fully  supported 
and  integrated  with  the  live,  virtual,  and  constructive  training  environments. 

Synchronize  scheduling  of  simulators,  training  tools,  and  packages  to  be  available 
to  train  for  first  deployment  to  ensure  full  system  capabilities  are  available  for 
operations. 
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Appendix  E. 
Acronyms 


AAN 

Army  After  Next 

ARL 

Army  Research  Laboratory 

BCTM 

Brigade  Combat  Team  Modernization 

BETS 

BCTM  Embedded  Training  Software 

CGF 

Computer  Generated  Forces 

CNATTU 

CNATT  Unit 

CNATT  DET 

Center  for  Naval  Technical  Training  Detachment 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DoD 

Department  of  Defense 

DSB 

Defense  Science  Board 

ECS 

Engagement  control  station 

FCS 

Future  Combat  Systems 

GAO 

Government  Accountability  Office 

GPR 

Ground-penetrating  radar 

GUI 

Graphical  user  interface 

HMDS 

Husky  Mounted  Detection  System 

HMMWV 

High  mobility  multipurpose  wheeled  vehicle 

ICC 

Information  and  Coordination  Central 

IDA 

Institute  for  Defense  Analyses 

IED 

Improvised  explosive  device 

KPP 

Key  Performance  Parameter 

LRIP 

Low  Rate  Initial  Production 

MANPRINT 

Manpower  and  Personnel  Integration 

MCAGCC 

Marine  Corps  Air  Ground  Combat  Center 

MDAP 

Major  Defense  Acquisition  Program 

MEADS 

Medium  Extended  Air  Defense  System 

MRAP 

Mine-Resistant  Ambush  Protection 

MRET 

MRAP  Egress  Trainer 

MTVR 

Medium  Tactical  Vehicle  Replacement 

NTSP 

Navy  Training  Systems  Plan 

ODS 

Operator  Driving  Simulator 

ORD 

Operational  Requirements  Document 

PAC 

Patriot  Advanced  Capability 

PCOFT 

Patriot  Conduct  of  Fire  Trainer 

SAM-D 

Surface-to-Air  Missile,  Development 

SDD 

System  Development  and  Demonstration 

TESS 

Tactical  Engagement  Simulation  System 

THAAD 

Terminal  High  Altitude  Air  Defense 

TPT 

Troop  proficiency  trainer 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

E-l 


TRL 

VBS2 


Technology  Readiness  Level 
Virtual  Battle  Space 
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effectiveness  with  training  strategies,  tools,  simulators  and  simulations  accommodated  as  an  integral  part  of  major 
Defense  programs. 
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